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Abstract 

Although  leadership  assessment  is  a  major  contributor  towards  Distinguished 
Graduate  (marking  a  student  as  the  top  ten  percent  of  ACSC  graduates),  the  school  has 
yet  to  formally  define  what  leadership  skills  or  behaviors  should  be  measured.  Indeed, 
the  Air  Force  itself  has  yet  to  define  what  it  takes  to  lead  today’s  airmen.  By  contrast,  the 
US  Army  provides  detailed  guidance  and  criteria  through  such  leadership  doctrinal 
products  as  FM  22-100.  The  purpose  of  this  study  was  to  determine  the  feasibility  of 
applying  some  part  of  the  Army’s  leadership  doctrine  to  ACSC’s  leadership  assessment 
program.  In  conjunction,  the  study  also  investigated  “who”  represented  the  best  insight 
to  assess  those  skills  or  behaviors. 

Interviews  were  conducted  with  a  limited  number  of  students,  staff,  and  senior 
leaders  to  gain  feedback  on  what  skills  or  behaviors  were  important  for  leadership, 
whether  they  were  reflected  in  the  Army  doctrine,  and  who  at  ACSC  could  effectively 
assess  those  stated  skills  or  behaviors.  In  particular,  subjects  were  asked  to  review  the 
Army’s  latest  leadership  assessment  tool,  the  Leader  Azimuth  Check  inventory. 

Analysis  indicated  acceptance  by  all  three  test  groups  (students,  operations,  and 
academics)  of  the  Azimuth  in  general  and  specific  support  for  the  five  elements  of 
communication,  motivating,  decision-making,  building,  and  integrity.  However,  results 
were  less  conclusive  for  “who”  should  measure  leadership.  Recommendations  are  made 
to  incorporate  a  limited  set  of  elements  from  the  Azimuth  which  were  both  widely 


IX 


accepted  and  recognized  as  observable  but  to  leave  the  basic  structure  of  ACSC’s 
leadership  assessment  program  unchanged. 
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Chapter  1 


Introduction 


Leaders  don’t  just  appear — they  are  molded.  The  Air  Force  is  not  handed 
leaders,  we  must  develop  them.  It’s  a  tribute  to  our  ojficers  and 
noncommissioned  ojficer  that  they  have  done  just  that  for  almost  50  years;  not 
only  for  the  Air  Force,  but  for  the  nation  as  well.  Air  University  is  an  institution 
where  our  past  and  present  leaders  gather  with  our  leaders  of  tomorrow.  It’s 
here  that  we  try  to  give  our  officers  and  NCOs  the  tools  they  need  to  effectively 
lead  the  many  and  varied  organizations  we  place  in  their  charge.  No  other  Air 
Force  mission  is  more  important.  And,  none  has  the  potential  to  yield  so  many 
rewards.  Out  of  these  halls  will  come  tomorrow’s  Billy  Mitchells  and  Hap 
Arnolds.  They  are  our  hope  for  the  future,  not  only  for  the  Air  Force,  but  for  the 
Nation. 


— Former  Secretary  of  the  US  Air  Force,  Sheila  E.  Widnall 
Leadership  development  is  an  elusive  problem.  While  every  organization  likes  the  idea  of 
ensuring  strong  leadership  in  its  next  generation,  not  every  organization  seems  able  to  define 
exactly  what  constitutes  it.  The  Air  Force  is  no  different.  In  an  article  for  the  Air  Force  Times  in 
1996,  then  Secretary  of  the  Air  Force  Sheila  Widnall  stated  “In  the  past  year,  Gen  Ronald  R. 
Fogleman,  Chief  of  Staff,  and  I,  as  Secretary  of  the  Air  Force,  have  taken  a  series  of  mutually 
supportive  steps  to  ensure  the  best  possible  leaders  for  the  future  Air  Force.  Those  steps  deal 
with  the  selection,  education,  and  accountability  of  commander... The  focus  of  command  and 
leadership  extends  to  Squadron  Officer’s  School,  Air  Command  and  Staff  College,  Air  War 
College....”^  However,  exactly  what  these  schools  are  supposed  to  focus  on  is  still  up  in  the  air 
since  currently  there  is  no  definitive  Air  Force  leadership  doctrine.  This  is  particularly  a  problem 
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for  the  Air  Force’s  Air  Command  and  Staff  College  (ACSC)  and  its  leadership  assessment 


program. 

ACSC’s  current  leadership  assessment  system  attempts  to  recognize  leadership  ability 
through  the  allocation  of  “Leadership  Points”  or  LPs  by  peers,  course  instructors,  operation 
officers,  and  senior  staff.  While  extensive  detail  is  provided  on  the  administration  of  the 
program  (who  can  give  how  many  points  for  example),  very  little  is  provided  on  the  objective 
criteria  needed  to  actually  determine  or  measure  student  leadership.  While  this  may  seem 
surprising,  one  must  remember  the  Air  Force  itself  has  yet  to  provide  objective  criteria  for  the 
service  as  a  whole.  In  addition,  ACSC  is  an  academic  environment  with  somewhat  artificially 
created  leadership  opportunities  and  where  debate  continues  still  between  academics  and 
operations  on  what  student  attributes  are  important.  For  example,  does  a  student  exhibit  good 
leadership  by  participating  in  community  service  events  or  by  helping  fellow  students  with 
academics?  Should  one  carry  more  weight  if  both  are  considered  leadership? 

It’s  the  intent  of  this  paper  to  better  define  ACSC  leadership  objective  criteria  and  to  identify 
who  can  measure  them  effectively.  Towards  this  end,  this  study  looks  at  merging  US  Army 
leadership  assessment  initiatives  with  senior  leader  and  student  perspectives  in  an  attempt  to 
provide  a  beginning  foundation  for  ACSC’s  own  leadership  assessment  program. 

Notes 

'  Sheila  Widnall,  “Building  Leadership-Step  by  Step,”  Air  Force  Times  56,  no.  31  (March  4, 
1996):  29. 


2 


Chapter  2 


Literature  Review 

It  is  of  little  use  to  an  officer  to  know  that  he  will  better  his  “efficiency  report” 
if  he  shows  better  leadership,  more  force,  or  greater  initiative  if  he  does  not 
know  what  he  must  do  to  be  rated  “superior”  on  these  qualities. 

— American  Institute  for  Research 
Development  of  a  Procedure  for  Evaluating  Officers  in  the  US  Air  Force,  1948 

This  review  presents  research  from  various  leadership  studies  to  answer  two  basic 
questions — what  leadership  skills,  behaviors,  or  traits  should  be  measured  for  field  grade 
officers  attending  ACSC,  and  who  should  measure  them.  To  identify  what  to  measure, 
recently  completed  doctrinal  research  and  initiatives  by  the  US  Army  are  explored.  To 
define  who  should  make  the  observations,  multi-rater  feedback  is  investigated.  First 
however,  a  review  of  ACSC’s  current  assessment  program  and  the  problems  leading  to  the 
two  aforementioned  questions  is  accomplished. 

ACSC  Assessment 

ACSC’s  Distinguished  Graduate  program  recognizes  a  student’s  abilities  in  both 
academics  and  leadership,  placing  him  or  her  in  the  top  ten  percent  of  an  already  elite  group. 
As  such,  it  becomes  a  delineating  factor  in  an  officer’s  record,  affecting  both  promotion  and 
further  professional  military  education.  Because  of  this,  it’s  important  to  understand  exactly 
how  the  two  critical  parts  interact  to  achieve  that  final  assessment. 
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As  the  program  currently  stands,  cumulative  Grade  Point  Average  (GPA)  points  are 
added  to  Leadership  Points  to  arrive  at  a  weighted  total  representing  a  “whole  person”  score/ 
Looking  first  at  GPA,  current  data  for  the  class  of  1999  indicates  over  95%  of  GPA  scores 
range  from  a  B  (scored  a  3.0)  to  an  A  (scored  a  4.0.)^  Therefore,  for  a  3-credit  class,  the 
GPA  point  spread  would  normally  range  from  a  9.0  (B)  to  a  12.0  (A.)  Totaling  the  number 
of  courses  offered  (9  plus  a  6-credit  research  or  elective  course),  the  spread  is  extended  to  a 
low-end  GPA  point  total  of  99  for  all  Bs  to  a  high-end  GPA  total  of  132  for  all  As.^  Thus,  33 
points  distinguish  low-end  and  high-end  academic  performers.  Basically,  because  ACSC 
rarely  uses  the  entire  GPA  scale  (0.0  to  4.0)  it  has  relegated  academics  to  only  a  one-grade 
difference  (B  to  an  A)  for  use  in  distinguishing  student  performance.  Thus  the  600-member 
student  body  is  stacked  into  a  rather  narrow  distribution  offering  little  distinction  in 
identifying  the  best  of  the  best. 

Turning  to  LPs,  the  data  indicates  over  90%  of  the  time  course  instructors  give  three 
students  0.2  LPs  (the  instructor  can  allocate  LPs  three  ways  -  0.3,  0.2,  0.1  or  0.2,  0.2,  0.2,  or 
0.3,  0.3.)"^  An  additional  0.1  can  be  allocated  based  on  peer  reviews.^  Therefore,  the 
leadership  point  spread  for  any  particular  course  generally  runs  from  0.0  to  0.3.  Since  their 
are  1 1  courses  involving  LPs  (operation  officer  and  senior  leadership  points  are  ignored  in 
order  to  stay  with  the  “average”  student  and  what  he  or  she  has  a  reasonable  chance  to 
influence  within  the  seminar)  a  student  can  earn  3.3  LPs.^  As  a  side  note,  each  student  starts 
with  2.0  LPs,  but  since  these  are  given  to  every  student,  they  do  not  affect  the  point  spread 
(see  Appendix  A  for  example  DG  scoring.)  As  such,  this  discussion  excludes  them.  Since 
LPs  carry  the  weight  of  a  nine-credit  course,  the  final  point  spread  can  range  from  0.0  to 
29.7.  When  you  combine  the  GPA  point  spread  and  the  LP  point  spread,  you’ll  find  a  total 
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variance  of  62.1 .  Thus  for  the  average  student  with  no  duties  outside  the  seminar  room,  LPs 
contribute  over  47%  of  the  final  point  spread  which  will  determine  who  is  a  DG  and  who  is 
not.  For  those  students  who  earn  LPs  from  operation  officers  or  senior  leadership,  LPs  will 
carry  even  more  weight. 


Table  1:  Maximum  Point  Spread  (based  on  11  courses) 


Instructors 

Peers 

Ops  Officers 

Senior  Leadership 

Point  Spread 

11  X  0.3=  3.3 

11  x0.1=  1.1 

4  offerings  x  0.3  =  1.2 

2  offerings  x  0.6  =1.2 

Oto  (9x6.8)  =  61.2 

Source:  ACSC  01  36-108,  Resident  Evaluation  Program  (15  Jul  98) 


With  the  importance  tied  to  assessing  leadership  at  ACSC,  one  would  assume  an 
established  set  of  criteria  exists  for  those  instructors,  peers,  operation  officers,  and  senior 
staff  to  use  as  they  differentiate  among  students.  Unfortunately  though  this  isn’t  the  case,  as 
the  lack  of  Air  Force  doctrine  detailing  what  the  service  itself  considers  as  leadership 
complicates  the  process.  As  such,  this  study  turns  to  the  Army  and  its  leadership  doctrine 
(based  on  hierarchical  leadership)  in  an  attempt  to  build  a  foundation  for  ACSC  leadership 
assessment. 


Hierarchical  Leadership 

Hierarchical  leadership  theory  attempts  to  model  leadership  requirements  by  asking  two 
basic  questions:  1)  what  leadership  requirements  are  required  within  an  organization  and  2) 
how  does  the  importance  given  to  those  various  requirements  change  as  an  individual 
progresses  through  the  various  divisions  or  levels  within  that  organization.  For  example,  if 
technical  competence  and  communication  skills  are  considered  important,  will 
communications  skills  become  more  or  less  in  demand  as  that  individual  climbs  within  the 
organization?  Will  the  importance  of  technical  competency  decrease,  remain  the  same,  or 
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increase?  As  such,  it  begins  to  narrow  the  search  for  leadership  criteria  by  first  focusing  on 
specific  levels  within  an  organization. 

Hierarchical  modeling,  with  its  emphasis  on  levels,  would  seem  to  lend  itself  to  the 
highly  structured  military  environment  and  indeed  several  studies  have  made  that  connection. 
Of  particular  interest  is  the  Stratified  Systems  Theory. 

Stratified  Systems  Theory 

Stratified  Systems  Theory  (SST)  is  a  variant  of  the  three-level  models  that  have  found  a 
popular  following  in  several  articles  on  organizational  structure.  Based  on  a  structure 
divided  horizontally  into  functional  departments  and  vertically  into  the  three  levels  of 
worker,  manager,  and  executive,  three-level  models  attempt  to  show  distinct  and  varied 

•y 

leadership  requirements  based  on  the  vertical  level.  For  example,  while  squadron-level 
pilots  and  maintainers  are  parts  of  different  functional  departments,  they  are  both  still 
considered  primary  operators,  performing  the  primary  work  of  the  organization. 
Theoretically,  their  leadership  requirements  would  vary  little.  However,  should  one  rise 
above  the  squadron  level  to  the  headquarters  level  command,  the  model  dictates  his  or  her 
leadership  requirements  must  change  as  that  individual  transitions  to  a  different  level 
(manager  rather  than  worker)  with  different  requirements  (supervising  rather  than 
performing.) 

Building  on  this.  Dr.  T.  Jacobs  (now  at  the  Industrial  College  of  the  Armed  Forces)  and 
Dr.  Elliott  Jaques  refined  three-level  modeling  into  the  SST.  Starting  with  the  simple 
concept  that  hierarchical  organizations  such  as  the  military  require  different  leadership 
abilities  depending  on  position,  Jacobs  and  Jaques  broke  those  positions  down  on  the  basis  of 
one  metric,  complexity  (measured  as  the  span  of  time  taken  to  complete  a  project  due  to  the 
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observation  more  complex  tasks  require  longer  periods  to  complete.)  With  this,  Jacobs  and 
Jaques  refined  their  three  primary  categories  (direct,  operation,  and  strategic)  into  seven 

o 

strata  identifying  specific  skills  needed  at  each  level  (Appendix  B.) 

The  Direct  domain  encompasses  those  individuals  tasked  with  accomplishing  the  day-to- 
day  mission  and  as  such  are  required  to  have  specialized  skills.  The  Operation  domain 
encompasses  two  leadership  roles.  First  is  the  “line”  supervisor  with  the  authority  and 
responsibility  for  overseeing  that  primary  functions  of  the  organization  are  completed. 
Second  is  the  senior  “staff’  officer,  providing  advice  and  assistance  to  the  line  manager  but 
lacking  the  authority  to  force  direct  action.  The  Strategic  domain  encompasses  those 
individuals  required  to  merge  internal  environment  requirements  with  those  outside  the 
organization  such  as  Congress  and  the  media.  Leaders  in  this  domain  will  ensure  the  health 
of  an  organization  from  5  to  50  years  out.^ 

SST  presents  an  empirically  based,  fully  developed  hierarchical  leadership  model 
describing  how  an  individual’s  ability  to  handle  complexity  comes  into  play  as  he  or  she 
moves  up  the  corporate  ladder.  As  such,  it  begins  to  narrow  the  search  for  leadership  criteria 
by  first  focusing  on  what  level  the  particular  target  group  is  operating  on.  For  this  research, 
the  question  would  become  “what  level  would  majors  attending  ACSC  need  to  operate  at 
upon  graduation?”  Once  this  is  answered,  than  the  field  should  be  defined  enough  to  turn  to 
look  at  what  specific  skills  and  behaviors  are  typical  at  that  level.  This  is  the  approach  the 
Army  took  several  years  ago  and  their  work  provides  a  beginning  foundation  for  ACSC  to 
draw  upon. 
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The  United  States  Army 

Nearing  the  eompletion  of  a  fundamental  shift  in  leadership  development  and 
assessment,  the  Army  has  taken  significant  strides  in  defining  leadership  doctrine  based  on  a 
large  part  on  the  ideas  espoused  by  the  SST.  The  final  stage  of  this  makeover  is  now  taking 
place  as  the  Army  takes  its  first  steps  towards  assessing  the  specific  skills  and  behaviors  it 
expects  throughout  its  various  leadership  levels.  A  key  player  in  this  effort  is  their  latest 
assessment  tool,  the  Leader  Azimuth  Check  inventory.  As  this  tool  sets  clear  and  observable 
leadership  criteria  for  majors  at  the  Army’s  equivalent  to  ACSC  (Command  and  General 
Staff  Officer  Course  or  CGSC),  it’s  worth  a  look  at  how  it  was  developed  and  whether  it  has 
applicability  for  ACSC. 

In  1995,  the  United  States  Army  kicked  off  a  major  initiative  to  remake  itself  into  a  force 
ready  for  the  challenges  of  the  2L'  century  -  Force  XXL  Part  of  this  remake  was  a  renewed 
interest  in  defining  leadership  doctrine.  Towards  this  end,  the  Center  for  Army  Leadership 
(CAL)  at  Fort  Leavenworth,  Kansas  was  tasked  as  the  lead  agency  to  “orchestrate  the 
development,  execution,  and  evaluation  of  current  and  future  leadership  and  leader 
development  initiatives  across  the  Army.”^*'  One  such  initiative  was  a  total  overhaul  of  the 
Army’s  cornerstone  piece  of  leadership  doctrine,  FM  22-100. 

Doctrine 

Taking  a  page  directly  from  Jacobs  and  Jacques,  the  Army  restructured  FM  22-100 
around  three  levels  of  leadership  as  distinguished  by  the  ability  to  handle  complexity.  Each 
level  is  generally  associated  with  various  command  structures  (company,  brigade,  etc.)  but  is 
not  linked  to  rank  (the  reason  being  that  while  a  major  at  the  battalion  level  may  work  at  a 
direct  level,  that  same  major,  placed  at  the  headquarters  level,  might  work  at  the 
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organizational  level.)  As  such,  the  position  occupied  is  the  predominate  factor  in  assessing 


what  leadership  skills  are  needed. 


Increased 

Uncertainty 

and 

Complexity 

NCA 

DOD 

CJCS 

CINC 

DA 

MACOM 
Component  CC 
Numbered  Army 
CORPS 
Division 
Brigade 
Battalion 
Company^ 

Reduced  - 

Uncertainty 

and 

Complexity 


'Global/Regional 

National/Societal 

Perspective 

Predominately  “Improving/ 
Building”  in  nature 


Direct  Leadership 

•Cohesion 

•Procedures 

•Motivation 


•Organizational/Systems 
and  Processes  Perspective 
•Predominately  “Operating/ 
Maintaining”  in  nature 


•Individual/Small  Group 
Task  Orientated  Perspective 
•Predominately  “Influencing 
and  Interpersonal”  in  nature 


Figure  1:  The  Levels  of  Army  Leadership 

Direct  leadership  is  the  “face-to-face,  front-line  leadership.” Here,  the  leader’s  span  of 
control  is  limited  to  those  that  he  or  she  can  reach  out  and  affect  directly,  and  the 
environment  is  one  of  more  certainty  and  less  complexity.  Operating  for  the  most  part  in 
accordance  with  stated  procedures,  the  leader  can  relatively  quickly  assess  what’s  working 
and  what’s  not  and  change  the  organization  as  needed.  Moving  up  the  complexity  ladder, 
organizational  leadership  focuses  on  “influencing  several  hundred  to  several  thousand 
people  indirectly  through  multiple  layers  of  subordinates.”  Here,  leaders  set  policies  and 
affect  the  environment  for  direct-level  leaders.  Because  of  the  additional  layers  between  the 
leader  and  those  who  he  or  she  affects,  results  take  longer  to  materialize  and  uncertainty 
increases.  Finally,  strategic  leadership  focuses  on  preparing  the  future  of  organizations  with 
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several  thousand  to  hundreds  of  thousands  of  people.  Problems  are  complex  and  often 


affected  by  organizations  outside  the  Army  (Congress,  interservice  cooperation,  etc.)  Thus, 
strategic  leaders  look  to  affect  such  items  as  force  structure,  allocation  of  resources,  and 
setting  a  strategic  vision.  As  with  the  SST,  the  Army  measures  complexity  levels  through 
time  (see  Appendix  C.) 

With  this  framework  set,  the  question  once  again  turns  to  “what”  to  measure.  Towards 
this  end,  the  Army  has  put  forth  four  categories  of  skills  a  leader  must  know  -  interpersonal, 

conceptual,  technical,  and  tactical,  and  three  sets  of  actions  a  leader  must  do  -  influence, 

1  -2 

operate,  and  motivate.  While  these  broad  categories  apply  to  all  three  of  the  previous 
levels,  different  subsets  of  skills  and  actions  are  needed  from  one  to  the  next.  For  example, 
while  FM  22-100  specifies  communicating,  team  building,  supervising,  and  counseling  as 
interpersonal  skills  for  the  direct  leader,  it  specifies  understanding  soldiers  and  supervising  as 
necessary  skills  for  the  organizational  leader. 


Table  2:  Leadership  Skills  by  Level 


Interpersonal 

Conceptual 

Technical 

Tactical 

Direct 

Communicating 

Team  Building 
Supervising 

Counseling 

Critical  Reasoning 
Creative  Thinking 

Moral  Reasoning 

Knowing 

Equipment 

Operating 

Equipment 

Doctrine 

Field  Craft 

War  Fighting 

Organizational 

Understanding  Soldiers 
Supervising 

Intent 

Filtering  Information 
Systems  Understanding 

Resourcing 

Synchronization 

Orchestration 

Strategic 

Communicating 

Negotiating 

Achieving  Consensus 
Staff  Building 

Envisioning 

Frame  of  Reference 
Development 

Dealing  with 

Uncertainty/ Ambiguity 
Developing  Intellectual 
Capital 

Strategic  Art 
Leveraging 
Technology 
Translating 
Political  Goals 
into  Military 
Objectives 

N/A 

Source:  Army  Field  Manual  22-100,  “Army  Leadership,”  4-1  through  7-15. 
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While  developing  the  right  skills  at  the  appropriate  level  of  leadership  is  important, 

leadership  doesn’t  begin  until  the  leader  applies  those  skills  and  acts.  Here  the  Army 

concentrates  on  the  three  aforementioned  categories:  influencing,  operating,  and  improving, 

which  are  further  broken  down  as  follows: 

Influencing  Operating  Improving 

Communicating  Planning  Developing 

Decision  Making  Executing  Building 

Motivating  Assessing  Learning. 

Leaders  at  all  stages  are  expected  to  perform  these  actions  but  not  at  the  same  complexity 

level.  While  the  direct  leader  is  expected  to  concentrate  on  building  teams,  the 

organizational  leader  will  be  expected  to  draw  on  his  or  her  additional  resourcing  skills  to 

build  a  network  of  teams  through  task  organization  and  resource  allocation.  Thus,  to  build  a 

framework  for  defining  the  “what”  of  leadership,  a  distinction  must  be  made  on  what  level  of 

leadership  or  situation  one  is  addressing.  In  this  case,  the  situation  is  ACSC. 

Because  of  the  tremendous  variety  between  follow-on  jobs,  it’s  impossible  to  make  an 
all-inclusive  statement  on  what  level  ACSC  graduates  should  operate  on.  However,  two 
facts  are  constant — ACSC  is  a  command  and  staff  institution  tasked  with  providing  a  cadre 
of  trained  officers  ready  to  serve  in  positions  of  higher  authority  and  responsibility  and  its 
graduates  are  considered  in  the  top  eighteen  percent  of  the  Air  Lorce.^^  As  such,  it’s 
appropriate  (and  should  be  expected)  to  demand  more  from  these  individuals  than  the 
standard  population  such  as  accelerated  staff  duty  at  the  unified  command  level  or  higher.  As 
such,  this  research  concentrates  on  the  SST’s  operational  domain.  The  overall  result  should 
be  aimed  at  producing  an  officer  ready  to  meet  the  complexities  of  working  on  a  staff  tasked 
with  creating,  balancing,  directing,  and  leading  multiple  projects  and  equipped  with  the 
ability  to  broaden  his  or  her  horizon  to  deal  with  a  longer  term  impact. 
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The  Army  uses  this  approach  at  CGSC  as  it  focuses  it’s  leadership  development  towards 
the  organizational  level  but  it  also  adds  a  futuristic  reason  as  well.  Using  inputs  from  Force 
XXI,  Army  After  Next,  and  Army  Vision  2010  initiatives,  CAL  sees  a  future  where 
operations  take  place  in  highly  ambiguous  situations;  where  information  technology  will,  at 
times,  provide  information  overload;  and  where  the  tactical,  operational,  and  strategic 
domains  begin  to  fuse  as  the  complexity  and  pace  of  operations  increase  dramatically.^^ 
Organizations  become  flatter,  increasing  span  of  control.  As  such,  leaders  will  have  to 
develop  the  critical  and  creative  thinking  skills  which  were  reserved  in  the  past  for  those  of 
higher  rank.  Thus  the  major  will  be  asked  more  and  more  to  step  away  from  the  direct 
domain  of  leadership  and  step  into  the  organizational  world. 

With  this,  the  focus  becomes  assessing  leadership  while  keeping  an  eye  towards  the 
operational  domain  (Army’s  organizational  level.)  One  instrument  that  can  provide  aid  in 
that  direction  is  the  Leader  Azimuth  Check  inventory  (Azimuth.)  The  Azimuth  is  the  latest 
assessment  tool  produced  by  the  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI.)  While  its  final  purpose  is  touted  as  allowing  leaders  at  all  levels  to  assess 
their  abilities  (it  focuses  on  the  three  categories  of  actions  all  leaders  perform),  it’s  current 
focus  is  to  provide  feedback  to  students  attending  the  CCSC.  As  such,  it  stands  to  reason 
that  some  items  might  be  transferable  to  ACSC  as  the  environments  are  much  the  same  as 
well  as  the  general  population. 

Leader  Azimuth  Check 

Originally  conceived  as  a  derivative  of  the  Army’s  assessment  tool  for  senior  leadership 
(Strategic  Leader  Development  Inventory  or  SLDI),  it  should  be  noted  that  the  current 
Azimuth  has  undergone  extensive  modification  to  arrive  at  its  current  version.  While  the 
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SLDI  was  developed  in  part  by  the  same  individuals  responsible  for  the  SST  (primarily,  Dr 
Jacobs)  which  the  Army  now  bases  so  much  of  its  doctrine  on,  questions  were  raised 
surrounding  the  factor  analysis  used  in  building  its  structure.  At  the  heart  of  the  problem  was 
the  issue  that  Jacobs  used  separate  and  fundamentally  different  surveys  to  collect  data  on 
leadership  skills  and  behaviors  based  on  whether  a  superior,  subordinate,  peer,  or  the  senior 
leader  himself  was  interviewed.  Researchers  at  the  Center  for  Army  Leadership  at  Ft 
Leavenworth  felt  that  building  a  set  of  competencies  based  on  the  resulting  data  was  similar 
to  comparing  apples  to  oranges  since  interview  questions  had  been  changed  between  target 
groups.  As  a  result,  they  began  to  look  at  constructing  an  assessment  tool  of  their  own, 
taking  what  parts  they  could  from  the  SLDI  but  modifying  it  to  meet  the  needs  of  direct  and 
operational  leaders  as  well.  In  1994,  they  began  by  using  data  collected  from  the  Army’s 
Combined  Arms  and  Services  Staff  School  (CASS)  using  only  the  peer  and  self-analysis 
tools  from  the  original  SLDI.  With  this  narrowed  field,  CAL  and  the  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  began  to  examine  in  detail  what 
changes  were  needed.  Thus  some  SLDI  items  were  eliminated  as  weak  items  while  others 
were  added  from  the  Army’s  core  competencies  which  seemed  to  strengthen  the  developing 
factor  structure  from  the  CASS  data.  The  result  was  Version  I  of  the  Azimuth  with  twelve 
elements  of  leadership  incorporating  98  observable  leadership  skills  or  behaviors. 

However,  as  noted  before,  the  Army  began  to  remake  itself  when  Force  XXI  kicked  off 
in  1995.  As  a  result,  the  Azimuth  was  modified  once  again  to  reflect  the  leadership  actions 
listed  in  the  new  FM  22-100.  With  this  new  direction,  CAL  refocused  to  build  a  concept- 
driven  product  rather  than  a  data-driven  one.  As  such  the  new  structure  forced  it  to  discard 
the  major  elements  of  Version  I;  however,  CAL  did  salvage  those  items  which  were  strongly 
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supported  by  previous  data  and  could  be  linked  to  the  new  guidance.  The  resulting  structure 
now  reflects  72  items  versus  the  original  98  and  has  been  simplified  for  administration  and 
scoring.  Both  versions  are  included  in  Appendices  D  and  E. 

Unfortunately,  because  only  one  CGSC  class  has  used  the  Azimuth,  more  validity 
testing  still  needs  to  be  completed  for  the  entire  instrument  to  become  universally  accepted. 
However,  initial  data  looks  strong  form  the  first  class  of  majors  and  supports  the  possibility 
of  adapting  parts  of  it  to  ACSC  (reference  Appendix  F  for  initial  data.) 

Summary 

Leadership  hierarchies  are  nothing  new  in  the  military.  Its  entire  structure  is  in  fact 
based  on  them  (take  for  example  senior  officers  overseeing  field  grade  officers  overseeing 
company  grade  officers.)  Therefore,  the  Army’s  initiatives  to  utilize  the  theory  behind  such 
hierarchies  as  three-level  modeling  (specifically,  the  Stratified  Systems  Theory)  shouldn’t  be 
surprising.  Indeed,  their  work,  including  the  development  of  leadership  criteria  in  the 
Azimuth,  should  provide  a  starting  point  for  ACSC  to  evolve  it’s  own  set  of  criteria  for  use 
as  a  baseline  for  deciding  who  deserves  the  additional  recognition  inherent  in  leadership 
points. 


Who  Measures 

While  making  a  decision  of  what  to  measure  is  a  significant  step  in  leadership 
assessment,  it’s  only  one  side  of  the  coin.  It  still  has  to  be  established  who  can  most 
effectively  measure  leadership  development  in  ACSC  students.  Is  the  student  a  better  judge 
of  his  abilities  or  are  those  around  him  such  as  peers  and  course  instructors?  Can  the  system 
utilize  just  one  input  source  such  as  instructors  or  operations  officers  or  is  there  an  under- 
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riding  bias  present  in  the  pull  between  academics  and  operations  which  makes  it  necessary  to 
provide  for  multiple  rating  sources  to  gain  an  objective  picture?  To  help  answer  these  types 
of  questions,  this  review  turns  to  look  at  the  concept  of  multi-rater  feedback. 

Multi-Rater  Feedback 

As  part  of  a  booming  leadership  development  industry,  multi-rater  feedback  has  made 
significant  inroads  into  more  than  90  percent  of  Fortune  1000  companies,  and  the  reason  is 
simple  -  it  provides  individuals  with  a  better  picture  of  their  own  skills  over  the  typical 

1  o 

supervisor-subordinate  assessment.  ACSC  itself  has  caught  the  fever,  as  its  current 
leadership  assessment  program  incorporates  feedback  from  four  separate  areas  (peers, 
instructors  operations  officers,  and  senior  leaders.)  However,  is  all  this  feedback  truly  worth 
the  effort  of  adding  these  additional  sources,  and  if  so,  are  there  any  special  considerations 
surrounding  this  type  of  feedback? 

The  Need 

To  answer  the  question  of  whether  additional  feedback  is  worth  the  effort,  this  study 
turns  to  one  of  the  largest  studies  conducted  to  determine  how  effectively  managers  rate  their 
own  skills.  In  1988  Harris  &  Schaubroeck  bought  together  a  total  of  36  independent  self¬ 
supervisor,  23  independent  peer-supervisor,  and  11  independent  self-peer  rating  studies. 
While  they  found  peers  and  superiors  often  agreed  on  an  individual’s  skills,  they  found  the 
same  couldn’t  be  said  about  the  individual  himself  as  compared  to  those  same  coworkers  and 
superiors. In  fact,  they  found  only  a  third  of  managers  produced  self-ratings  that  matched 
what  others  had  to  say.  Of  those  who  missed  the  mark,  a  third  had  an  over-inflated  view  of 

90 

their  skills  while  the  remaining  third  underrated  their  abilities. 
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But  how  damaging  is  it  really  if  a  manager  misses  the  mark  on  assessing  his  or  her  own 
abilities?  Ellen  Van  Velsor,  a  researcher  for  the  Center  for  Creative  Leadership  (whose 
customers  include  IBM,  Nike,  and  the  Army)  let  coworkers  weigh  in  on  that  question.  For 
the  two  thirds  of  the  leaders  who  had  distorted  views  of  themselves,  the  most  damaging  as 
perceived  by  coworkers  was  the  over  rater  (self-doubters  were  actually  scored  higher  on 
effectiveness  than  both  the  other  groups.)  Unfortunately,  Van  Velsor  also  found  this 
problematic  group  of  over  raters  grew  in  numbers  as  she  looked  higher  in  various 
organizations.  Thus  it  would  seem  feedback  becomes  even  more  critical  as  one  advances  as 
success  often  lures  individuals  into  a  false  sense  of  confidence  about  their  own  abilities  at  a 
time  when  their  affect  on  the  organization  is  increasing. 

However,  how  much  of  a  problem  can  this  become?  Won’t  the  organization  itself 
eventually  recognize  problem  supervisors  and  remove  them?  The  answer  is  “no”  as  the 
organization  itself  is  often  relying  on  the  same  documented  feedback  from  superiors  that  the 
individual  is.  Take  for  example  research  by  Lieutenant  General  Walter  F.  Ulmer,  Jr.  (USA, 
Ret),  Distinguished  Fellow  of  the  US  Army  War  College  and  former  nine  year  President  and 
CEO  of  the  Center  for  Creative  Leadership.  In  an  article  published  for  the  Spring,  1998  US 
Army  War  College  Quarterly  he  states: 

“If  one  were  to  query  serving  officers  about  the  percentage  of  battalion 

brigade,  division,  and  corps  commanders  who  were  seen  as  unsatisfactory 

leaders  by  a  plurality  of  their  subordinates  and  by  many  of  their  peers,  I 

suspect  the  figure  would  be  between  15  and  25  percent.  Actually,  there  have 

been  enough  informal  surveys,  anecdotal  reports,  and  ancillary  studies  over 

21 

the  past  20  years  to  make  this  more  than  a  ‘suspicion.  ’  ” 

The  supporting  data  Ft  Gen  Ulmer  refers  to  is  an  Army  War  College  Military  Studies  Project 
authored  by  Tilden  Reid  entitiled  “Performance  of  Successful  Brigade  Commanders  Who 
Were  Selected  to  BG  as  Viewed  by  Their  Former  Battalion  Commanders,”  which  concluded 
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that  28  percent  of  those  brigadier  generals  should  not  have  been  selected  as  viewed  by  their 
22 

subordinates. 

So,  if  single-source  feedback  isn’t  working,  can  multi-rater  feedback  fill  the  void  in  a 
military  environment?  It  seems  the  Army  thinks  so.  Currently,  multi-source  feedback  is 
used  as  part  of  the  evaluation  processes  at  Reserve  Officers’  Training  Corps  Advanced 
Camp,  the  US  Military  Academy,  Ranger  school,  the  Combined  Arms  and  Services  Staff 
School  (CASS),  the  Warrant  Officer  Staff  Course,  and  the  Army  Management  Staff  College. 
In  1993,  the  Army  moved  into  high  gear  as  it  commissioned  a  project  in  conjunction  with 
ARI  and  the  Center  for  Creative  Leadership  to  “explore  emerging  technologies  to  assess 
skills  required  for  successful  leader  performance  and  techniques  to  develop  leader  and 
decision  making  skills...”  One  of  the  recently  completed  milestones  of  this  commission 
has  been  the  initiation  of  multi-rater  feedback  at  CGSC  through  the  previously  mentioned 
Leader  Azimuth  Check  instrument.  Once  this  initial  research  is  completed,  the  Army 
intends  to  move  the  Azimuth  to  its  units  throughout  the  service  to  serve  as  a  basis  for  its 
leadership  development  program.  This  says  quite  a  bit  about  how  committed  they  are  to 
incorporating  multi-rater  feedback. 

Implementation  Considerations  and  Concerns 

However,  while  multi-rater  feedback  can  provide  invaluable  information  to  an 
individual  on  his  or  her  own  decision-making  skills,  there  are  some  critical  conditions  to  be 
met.  First  and  foremost,  a  relevant  assessment  instrument  based  on  accepted  organizational 
competencies  must  be  developed.  To  help  ensure  this,  Edwards  and  Ewens,  president  and 
CEO  of  TEAMS,  Inc  and  credited  with  coining  the  phrase  “360  degree  feedback”  suggest 
organizing  a  focus  group  to  answer  the  question  of  “What  are  the  critical  competencies  the 
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organization  will  need  in  the  future?”  While  no  means  a  formalized  process,  this  paper 
hopes  to  start  ACSC  on  this  process  by  interviewing  key  staff  members  and  students  in  the 
hopes  of  ascertaining  their  thoughts  on  the  necessary  leadership  skills  for  ACSC  graduates. 
Additional  concerns  surrounding  the  involvement  of  the  target  audience,  ensuring  a  process 
to  provide  confidentiality,  training  for  participants,  continuing  review  of  the  process,  and  a 
mechanism  for  follow-up  support  for  self-development  will  be  revisited  in  the  conclusions 
section  of  this  paper^^. 

Before  leaving  this  issue,  this  paper  would  be  remiss  if  it  didn’t’  address  the 
considerable  body  of  work  pointing  to  the  possible  dangers  of  using  multi-rater  feedback 
within  an  assessment  process.  Will  peers  who  are  competing  for  the  same  rewards  attempt  to 
undercut  one  another?  Will  the  assessment  become  nothing  more  than  a  popularity  or 
visibility  contest  with  participants  jockeying  for  position  instead  of  coming  together  and 
working  as  a  team?  While  there  is  resistance,  this  author  contends  a  well-developed  set  of 
competencies,  serving  as  a  guide  for  students  and  staff  who  provide  limited  feedback  to 
justify  their  ratings,  can  go  a  long  ways  to  minimizing  potential  problems.  In  addition, 
ensuring  multiple  inputs  from  each  source  of  feedback  will  dampen  the  problems 
encountered  by  the  occasional  jealous  co-worker.  For  a  seminar  at  ACSC  for  example, 
feedback  would  be  received  from  12  to  13  different  peers  limiting  the  effect  from  any  one 
individual  in  particular. 


Summary 

Effective  leadership  assessment  is  judged  by  how  well  it  provides  accurate  feedback  to 
both  the  individual  and  the  organization.  As  such,  many  are  beginning  to  look  at  multi-rater 
feedback  as  an  alternative  to  the  more  traditional  single-source  rating  systems  which  are 
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often  leaving  both  the  individual  and  the  organization  with  an  incomplete  picture.  It  fits 


particularly  well  within  ACSC  as  students  are  placed  within  multiple  command  chains  where 
no  one  individual  can  observe  all  of  a  student’s  actions.  With  this  then,  the  stage  is  set  for 
looking  at  who  should  participate  in  student  leadership  assessment  and  thus  the  focus  will 
now  turn  to  the  results  for  staff  and  student  interviews. 


Conclusion 

ACSC  places  significant  interest  in  bringing  leadership  assessment  into  its  overall  DG 
program.  As  such,  it  needs  to  ensure  objective  criteria  are  provided  to  both  staff  and  students 
alike  to  guide  them  in  their  decision  making  process.  Fortunately,  the  Army’s  Azimuth 
inventory  provides  a  starting  point  for  developing  those  criteria.  As  for  who  should  get  a  say 
in  using  that  criteria,  the  starting  focus  should  be  on  bringing  in  multiple  sources  in  order  to 
provide  feedback  from  all  areas  of  a  student’s  performance.  With  this  background  set,  this 
research  now  focuses  on  interviewing  senior  leadership  and  students  in  order  to  mold  the 
Azimuth  indicators  into  something  useable  at  ACSC,  and  to  determine  exactly  who  should 
participate  in  assessing  a  student’s  leadership  ability  and  to  what  extent  their  assessment 
should  play  in  the  overall  leadership  point  allocation. 
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Chapter  3 


Methodology 

True  genius  resides  in  the  capacity  for  evaluation  of  uncertain,  hazardous,  and 
conflicting  information. 

Sir  Winston  Churchill 

The  subjects  for  this  study  were  senior  leaders,  staff,  and  seminar  leaders  from  the  Air 
Command  and  Staff  College  at  Maxwell  AFB.  The  instrumentation  used  to  collect  the  data 
included  one-on-one  interviews  utilizing  questions  developed  by  the  author  and  approved  by 
ACSC’s  Evaluations  department  and  the  Leader  Azimuth  Check  inventory. 

Subjects 

The  subjects  represented  a  stratified  sampling  of  leadership  (n=18)  across  the  various 
functions  within  ACSC  including  academics,  operations,  the  commandant,  and  students.  Those 
interviewed  consisted  of  the  Commandant,  Division  1-4  Operations  Officers,  the  Dean  of 
Students,  the  chairs  of  the  three  major  academic  departments  responsible  for  ACSC  curricula — 
DBA,  DEB,  and  DEC,  and  finally,  nine  seminar  leaders  representing  both  semesters  of  the  class 
of  1999.  These  individuals  were  chosen  based  on  their  positions  that  afforded  them  unique 
perspective  into  the  overall  workings  and  underpinnings  of  ACSC.  All  individuals  outside  of  the 
students  had  at  least  two  years  experience  at  ACSC  and  all  had  served  as  course  instructors  thus 
yielding  an  equal  representation  between  students  (n=9)  and  academics  (n=9). 
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Instrumentation 


Instrumentation  used  for  this  study  included  interviews  with  staff  and  students  from  ACSC. 
During  the  interview,  subjects  were  asked  to  answer  seven  questions  contained  in  an  interview 
request  letter  (Appendix  H.)  In  addition,  each  individual  was  requested  to  review  the  72  items 
contained  in  the  Leader  Azimuth  Check  inventory.  Version  II  (Appendix  E.) 

Procedures 

Research  procedures  focused  on  determining  what  should  be  measured  for  leadership 
assessment  and  who  should  measure  it,  and  consisted  of  obtaining  interview  approval,  data 
collection  through  the  interview  process,  and  analysis.  To  obtain  approval  to  conduct  the 
interviews,  a  staff  summary  package  was  sent  to  ACSC’s  Evaluations  Department  and  approved 
by  each  of  the  staff  subjects  listed  earlier  (Appendix  H.)  Each  staff  member  was  then  contacted 
to  schedule  the  interview  itself.  Seminar  leaders  were  selected  via  line-of-sight  and  asked  to 
participate  when  their  schedule  allowed. 

The  20-30  minute  interviews  centered  on  two  products,  the  aforementioned  seven  questions 
and  the  Azimuth  inventory.  While  discussing  the  third  question,  a  copy  of  the  Azimuth 
inventory  was  presented.  Subjects  were  then  asked  to  comment  on  whether  any  items  fit  their 
definition  of  leadership  and  to  mark  whether  those  items  were  observable,  somewhat  observable, 
or  not  observable  (if  time  was  short,  the  inventory  was  left  with  the  subject  to  complete  at  his  or 
her  leisure.)  The  interviewer  provided  no  additional  information  on  the  items  contained  in  the 
Azimuth. 

Qualitative  comments  from  the  interviews  were  collected  and  categorized.  They  were  then 
examined  to  determine  which  Azimuth  elements  were  appropriate  for  ACSC  leadership 
assessment  and  what  groups  should  be  included  in  that  process. 
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Chapter  4 


Results 

The  interview  process  for  this  study  consisted  of  two  parts.  The  first  was  a  series  of  seven  questions 
designed  to  answer  the  questions  of  what  should  form  the  basis  for  leadership  assessment  at  ACSC  and 
who  should  participate  in  the  assessment  program.  Incorporated  in  these  questions  was  an  additional 
emphasis  on  addressing  whether  the  Army’s  Leader  Azimuth  Check  could  serve  as  a  basis  for 
measurement.  Towards  this  end,  the  subjects  were  asked  to  comment  on  both  the  appropriateness  of  the 
Azimuth  items  based  on  their  defined  set  of  leadership  skills  and  behaviors  and  whether  the  items  could 
be  observed  in  an  ACSC  setting.  Overall  results  indicated  agreement  among  the  test  groups  on  “what”  to 
measure  and  acceptance  of  the  Azimuth  as  a  whole;  however,  opinion  was  divided  on  “who”  should 
participate. 

What  Skills  or  Behaviors 

Each  subject  was  first  asked  what  skills,  behaviors,  or  traits  they  considered  as  part  of  leadership. 
Detailed  results  from  this  question  are  contained  in  Appendix  I  and  show  heavy  consensus  towards  team 
work.  Next,  subjects  were  asked  whether  any  items  from  the  Azimuth  survey  fit  their  definition  of 
leadership.  Because  all  but  one  respondent  found  it  easier  to  answer  based  on  the  broader  categories  of 
elements,  the  data  is  presented  based  on  the  14  elements  instead  of  the  72  individual  items.  Below  are  the 
top  five  elements  selected  by  the  three  test  groups  (students,  operations,  and  academics.)  A  complete 
listing  of  all  responses  is  found  in  Appendix  J. 
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Top  Five  Selected  Azimuth  Elements 


u 

Communication 

Decision  Making 

Motivating 

Building 

Integrity 

mil  Seminar  Leaders  (n=9) 

8 

5 

6 

3 

6 

□  Comm/Operations  (n=5) 

4 

3 

4 

3 

1 

S  Dean/Faculty  (n=4) 

3 

3 

2 

4 

3 

B  Total  (n=  18) 

15 

11 

12 

10 

10 

Figure  2:  Top  Five  Azimuth  Elements  Selected  hy  Subjects 

A  corollary  to  the  question  of  “what  to  measure”  is  “can  it  be  measured?”  As  such,  each  subject  was 
asked  to  assess  whether  each  of  the  72  items  within  the  Azimuth  were  “Observable,”  “Somewhat 
Observable,”  or  “Not  Observable.”  Because  of  the  size  of  data  representation,  all  results,  including  the 
overall  results,  are  included  in  Appendix  K. 


Who  Should  Measure 

Subjects  were  asked  to  provide  their  opinions  on  who  could  effectively  measure  those  skills  or 
behaviors  they  listed  as  important  in  leadership  assessment  at  ACSC.  Results  from  the  three  respondent 
groups  and  overall  totals  are  depicted  below.  Once  “who”  was  established,  subjects  were  asked  to  weight 
the  inputs  each  source  should  have.  Results  concerning  appropriate  weighting  were  as  varied  as  the 
number  of  subjects  and  no  definite  conclusions  can  be  drawn  from  the  data.  However,  the  second  graph 
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depicted  below  indicates  support  for  maintaining  the  current  grouping  of  peers,  course  instructors, 
operation  officers,  and  senior  leaders  as  the  appropriate  mix  for  assessing  leadership  at  ACSC.  Complete 
results  are  listed  in  Appendix  L. 
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Figure  3:  Who  Should  Measure  -  Broken  down  to  reflect  individual  categories  (peers,  seminar 

leaders,  course  instructors,  etc) 
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Who  Should  Measure 
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Figure  4:  Who  Should  Measure 
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Chapter  5 


Conclusions 

Life  is  the  art  of  drawing  sufficient  conclusions  from  insufficient  premises. 

— Samuel  Butler 
Notebooks 

This  ehapter  diseusses  the  results  from  interviews  with  Air  Command  and  Staff  senior  leadership  and 
students  centered  on  the  two  questions  of  “what  leadership  behaviors  to  measure?”  and  “who  should 
measure  them?”  The  intent  is  to  relate  them  back  to  the  larger  issues  of  inserting  an  Army  doctrinally- 
based  product  as  a  foundation  for  determining  the  “what”  for  leadership  assessment  at  an  Air  Force 
intermediate  school  and  whether  multi-rater  feedback  can  be  beneficial  at  that  same  school.  It  concludes 
with  recommendations  for  both  the  school  and  further  research. 

What  to  Measure 

Interview  data  indicate  a  consensus  on  “what”  should  be  measured  for  leadership  assessment  and 
further  indicate  that  parts  of  the  Army’s  Leadership  Azimuth  Check  Inventory  could  be  used  as  a  baseline 
for  student  leadership  assessment. 

While  the  subjects  varied  from  students  to  the  acting  Commandant,  some  central  themes  didn’t  - 
teamwork,  decision  making,  and  integrity  were  important.  Teamwork  especially  was  considered  critical, 
with  over  a  third  of  the  subjects  either  highlighting  organizing  and  working  in  teams  as  important  or 
highlighting  those  skills  they  thought  necessary  for  team  building  (such  as  communication)  as  important 
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(see  bolded  items  in  Appendix  1.)  Indeed,  the  phrases  team  work  and  team  building  came  up  so  often 
during  the  interviews  the  author  questioned  several  subjects  on  whether  this  had  been  discussed  at  earlier 
staff  meetings  or  at  other  such  gatherings.  The  response  in  all  cases  was  “no.”  Instead  the  reason  given 
on  a  universal  basis  was  the  subjects  realized  leaders  in  higher  positions  of  authority  no  longer  has  the 
luxury  of  controlling  projects  directly.  Rather,  they  now  had  to  work  through  others  in  order  to  ensure 
success. 

Turning  to  the  Leader  Azimuth  Check,  it  was  generally  well  received  with  only  5  of  the  total  75 
comments  indicating  a  need  for  eliminating  one  or  two  total  elements  and  only  6  of  the  total  75  indicating 
minor  modifications  (changing  or  adding  one  specific  item  within  an  element).  Not  surprisingly,  when 
subjects  were  asked  to  single  out  those  elements  that  most  effectively  assessed  leadership,  they  selected 
those  elements  most  closely  related  to  the  three  central  themes  of  teamwork,  decision  making,  and 
integrity  (Communication-83%,  Motivating-67%,  Decision  Making-61%,  Building-56%,  and  Integrity- 
56%. )  The  next  elements  most  singled  out  were  Respect  and  Selfless  Service  at  33%  followed  by 
Executing  at  28%,  Learning,  Planning/Organizing,  Assessing  and  Emotional  Stability  at  22%;  Developing 
at  17%,  and  Other  at  11%. 

While  the  Azimuth  was  well  received,  another  part  of  determining  what  leadership  skills  or  behaviors 
should  form  the  basis  for  assessment  is  whether  or  not  they  are  observable.  Looking  at  the  top  five 
elements  of  Communication,  Motivating,  Decision  Making,  Building,  and  Integrity,  Building  received  the 
top  marks  with  only  one  item  receiving  less  than  89%  observable.  Overall,  it  averaged  88%  for  its  five 
items  with  the  lowest  item  weighing  in  at  50%.  Communication  was  next  with  an  overall  average  of  73% 
with  the  lowest  item  also  weighing  in  at  50%.  Results  for  Motivation  were  mixed  with  three  items 
scoring  high  (average  76%)  and  two  others  scoring  low  (33%. )  Integrity  items  measured  68%  on  average 
with  no  item  scoring  less  than  56%.  Einally,  Decision  Making  received  poor  marks  overall  at  41%.  Of 
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the  other  elements,  Learning  (68%),  Respect  (76%),  Emotional  Stability  (94%),  and  Other  (75%)  all 
received  good  marks.  For  the  remaining  elements,  Planning/Organizing,  Executing,  and  Selfless  Service 
have  some  items  which  scored  above  50%  while  Assessing  was  a  wash  across  the  board  with  the  highest 
item  only  receiving  a  28%  vote  of  confidence. 

The  Azimuth  is  a  direct  reflection  of  the  revised  Army  doctrinal  categories  of  influencing 
(Communicating,  Decision  Making,  and  Motivating),  operating  (Planning,  Executing,  and  Assessing), 
and  improving  (Development,  Building,  and  Eearning)  and  its  overall  acceptance  lends  credence  to 
investigating  the  latest  addition  of  its  Army  leadership  doctrine,  EM  22-100  for  use  at  ACSC.  However, 
the  results  on  observability  indicate  adopting  any  part  of  it  will  have  to  be  considered  in  light  of  the 
school  environment. 


Who  Measures 

While  the  results  from  the  interviews  indicate  some  commonality  on  who  should  participate  in 
leadership  assessment  they  stop  there,  as  the  weighting  comments  are  as  varied  as  the  individuals.  Peers 
were  included  for  all  but  two  subjects  (one  seminar  leader  and  one  academic  member)  followed  next  by 
course  instructors  (14)  and  operations  officers  (13)  for  a  percentage  rating  of  89%,  78%,  and  72% 
respectively.  Senior  leadership  came  in  for  9  of  the  18  subjects  (50%)  and  two  subjects  reported  a  desire 
for  a  change  in  the  current  observers  with  one  bringing  in  seminar  leaders  and  another  requesting  the 
addition  of  a  permanent  faculty  member  attached  to  each  seminar.  As  for  the  grouping  of  observers, 
keeping  the  four  current  groups  (peers,  course  instructors,  operation  officers,  and  senior  leadership)  was 
reported  most  often  but  was  still  only  included  in  8  of  the  overall  18  interviews  (44%)  with  only  three 
subjects  reporting  the  same  weighting  among  this  grouping  (keeping  the  current  system  in  place  at 
ACSC.)  The  greatest  divergence  occurred  among  those  subjects  from  the  academics  specialty  with  all 
four  reporting  different  groupings. 
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Because  of  the  disparity  in  the  results  on  who  should  measure  leadership  at  ACSC  and  with  what 
weighting,  drawing  implications  from  them  is  difficult.  For  utilizing  multi-rater  feedback,  there  does 
seem  to  be  agreement  that  more  than  one  observing  group  is  needed  as  only  two  subjects  (both  seminar 
leaders)  responded  with  just  one.  Towards  this  end,  peers,  course  instructors,  and  operation  officers 
should  be  looked  at  as  the  top  contenders  to  be  included.  However,  because  of  the  differing  of  opinions 
of  the  subjects,  this  paper  will  not  make  any  attempt  to  exclude  any  one  group  or  to  define  any  weighting 
system.  If  ACSC  should  determine  a  set  of  critical  competencies  does  exist  for  its  leadership  assessment 
program,  further  review  should  follow  to  determine  if  this  might  provide  the  focus  needed  to  bring  theses 
groups  closer  together. 


Recommendations 

Because  the  focus  group  for  this  research  represented  such  a  small  total  percentage  of  the  overall 
ACSC  total  population,  the  reader  must  be  careful  before  drawing  any  sweeping  conclusions.  However, 
because  of  the  importance  assigned  leadership  assessment  at  ACSC  and  the  significant  support  for  the 
Leader  Azimuth  Check  inventory,  this  paper  recommends  ACSC  look  hard  at  implementing  a  beginning 
set  of  leadership  competencies  based  on  that  instrument.  In  particular,  emphasis  should  be  given  to  using 
those  items  within  the  elements  of  Communication,  Motivating,  Decision  Making,  Building,  and  Integrity 
that  correlated  well  with  the  results  on  observability. 

As  the  data  is  conflicting  on  exactly  who  should  assess  those  competencies,  this  paper  recommends 
leaving  the  current  system  in  place,  utilizing  peers,  course  instructors,  operation  officers,  and  senior 
leadership.  However,  changes  should  be  made  to  bring  it  in  line  with  the  multi-rater  concept.  First  and 
foremost,  those  involved  in  the  assessment  program  must  be  trained  on  what  to  assess  and  how  their 
assessments  affect  the  targeted  population.  For  the  peer  group,  training  could  be  provided  during  the 
beginning  blocks  of  the  Leadership  and  Command  curriculum  (this  is  the  first  block  of  instruction 
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provided  for  students  at  ACSC.)  As  for  the  other  three  groups,  the  Evaluation  department  of  ACSC 
should  take  the  lead  for  ensuring  they  have  the  appropriate  knowledge  of  the  criteria  and  its  affects,  as 
they  are  responsible  for  overseeing  student  evaluation  in  general.  Next,  confidentially  should  be  ensured 
through  the  continued  use  of  the  computerized  system  that  allows  the  peer  group  to  enter  inputs 
anonymously.  The  intent  is  to  allow  an  environment  that  minimizes  recrimination  within  a  group  of 
equals.  Looking  at  the  need  for  a  continuing  review  of  the  assessment  program  to  address  any  changes 
that  occur  within  the  ACSC  organization,  the  Commandant’s  staff,  which  is  comprised  of  senior  student 
population  under  the  direction  of  the  senior  staff  at  ACSC,  should  be  employed  to  review  the 
effectiveness  of  the  assessment  program  at  predetermined  times  throughout  the  year.  In  this  way,  not  only 
is  there  a  representation  of  the  targeted  population,  but  also  a  representation  of  the  key  decision  makers  at 
ACSC  in  general. 

Finally,  is  the  largest  issue  surrounding  multi-rater  feedback  and  assessment — using  it  for  further 
development.  In  Phase  III  of  the  current  Leadership  and  Curriculum  (which  is  placed  towards  the  end  of 
a  student’s  ACSC  experience),  students  are  required  to  develop  an  action  plan  designed  to  improve  their 
leadership  abilities.  However,  before  a  student  can  develop  a  plan  to  improve  himself,  that  student  must 
establish  a  launching  point.  As  such,  this  author  recommends  expanding  the  current  leadership 
assessment  program  to  include  written  comments  from  the  assessment  groups,  based  on  the 
aforementioned  leadership  criteria,  in  order  to  provide  that  student  a  better  picture  of  his  or  her  current 
abilities.  While  the  assessment  system  is  set  up  to  only  recognize  those  who  receive  leadership  points,  at 
least  some  of  the  student  population  would  arrive  at  Phase  III  with  hard  data  to  start  an  action  plan  on.  In 
addition,  should  additional  peer  assessments  be  requested  on  all  students  in  phase  III,  the  targeted 
population  would  have  experience  with  the  leadership  criteria  through  its  use  for  leadership  point 
allocation. 
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While  instigating  leadership  criteria  from  the  Army’s  leadership  doctrine  provides  an  initial  patch  to 
ACSC’s  leadership  assessment  program,  the  school  shouldn’t  stop  there.  The  final  recommendation  from 
this  author  is  continued  research  on  developing  leadership  doctrine  specific  to  the  Air  Force.  Towards 
this  end,  the  school  should  sponsor  further  research  to  poll  Air  Force  leadership  on  what  it  wants  from  its 
graduates  of  ACSC  as  well  as  sponsor  research  to  further  examine  if  the  Air  Force  could  capitalize  on  the 
Army’s  lead  on  leadership  doctrine.  If  ACSC  is  to  educate  majors  to  “lead  in  developing,  advancing,  and 
applying  aerospace  poser”  then  the  school  must  take  the  first  steps  to  articulate  the  objectives  involved  in 
meeting  that  goal. 
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Appendix  A 


Distinguished  Graduate  Scoring  (Examples) 


Student  A  (All  A’s7No  Leadership  Points) 

Course 

Flour 

s 

Acad 

Grade 

GPA 

Cum  GPA  Pts 
(hoursxGPA) 

Cum  GPA 

LP 

Cum  LPs 
2.00 

Leadership7Command 

N7A 

N7A 

N7A 

N7A 

N7A 

MS 

2.00 

Nature  of  War 

3 

A 

4.00 

12.00 

4.00 

MS 

2.00 

War  Theory 

3 

A 

4.00 

24.00 

4.00 

MS 

2.00 

Ops  Officer  Input  1 

MS 

2.00 

Strategic  Environment 

3 

A 

4.00 

36.00 

4.00 

MS 

2.00 

Operational  Forces 

3 

A 

4.00 

48.00 

4.00 

MS 

2.00 

Conflict  Resolution 

3 

A 

4.00 

60.00 

4.00 

MS 

2.00 

Ops  Officer  Input  2 

MS 

2.00 

Joint  Operations 

3 

A 

4.00 

72.00 

4.00 

MS 

2.00 

Air  and  Space  Ops 

3 

A 

4.00 

84.00 

4.00 

MS 

2.00 

Ops  Officer  Input  3 

MS 

2.00 

Leadership7Command 

3 

A 

4.00 

96.00 

4.00 

MS 

2.00 

Tandem  Challenge 

N7A 

N7A 

N7A 

N7A 

N7A 

MS 

2.00 

Force  2025 

3 

A 

4.00 

108.00 

4.00 

MS 

2.00 

Ops  Officer  Input  4 

MS 

2.00 

Research  Project 

6 

A 

4.00 

132.00 

4.00 

N7A 

2.00 

Senior  Leadership 

MS 

2.00 

Leadership 
Performance 
(Leadership  Points) 

9 

1 

18.00 

Note:  Because  2.00  LPs  are  given  by  default,  the  final  tally  reflects  at  least  18  points  for  every 
student.  If  these  are  removed  to  reflect  only  those  points  which  are  earned,  the  final  tally  becomes 
0.00  which  is  the  total  discussed  in  the  introduction. 

Whole  Person  Score  (WPS)  =  (Cum  LPs  +Cum  GPA  Pts)/Total  hours  (42) 

Student  A’s  WPS  =  (18.00  +  132)742  =  (150)742  =  3.57 
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Student  B  (All  B’s7Leadership  Points  from  lnstructors7Peers) 

Course 

Flour 

s 

Acad 

Grade 

GPA 

Cum  GPA  Pts 
(hoursxGPA) 

Cum 

GPA 

LP 

lnstuctor7Peer 

Cum  LPs 
2.00 

Leadership7Command 

N7A 

N7A 

N7A 

N7A 

N7A 

271 

2.30 

Nature  of  War 

3 

A 

3.00 

9.00 

3.00 

271 

2.60 

War  Theory 

3 

A 

3.00 

18.00 

3.00 

271 

2.90 

Ops  Officer  Input  1 

MS 

2.90 

Strategic  Environment 

3 

A 

3.00 

27.00 

3.00 

271 

3.20 

Operational  Forces 

3 

A 

3.00 

36.00 

3.00 

271 

3.50 

Conflict  Resolution 

3 

A 

3.00 

45.00 

3.00 

271 

3.80 

Ops  Officer  Input  2 

MS 

3.80 

Joint  Operations 

3 

A 

3.00 

54.00 

3.00 

271 

4.10 

Air  and  Space  Ops 

3 

A 

3.00 

63.00 

3.00 

271 

4.40 

Ops  Officer  Input  3 

MS 

4.40 

Leadership7Command 

3 

A 

3.00 

72.00 

3.00 

271 

4.70 

Tandem  Challenge 

N7A 

N7A 

N7A 

N7A 

N7A 

271 

5.00 

Force  2025 

3 

A 

3.00 

81.00 

3.00 

271 

5.30 

Ops  Officer  Input  4 

MS 

5.30 

Research  Project 

6 

A 

3.00 

99.00 

3.00 

N7A 

5.30 

Senior  Leadership 

MS 

5.30 

Leadership 
Performance 
(Leadership  Points) 

9 

47.70 

-  Note:  Because  2.00  LPs  are  given  by  default,  the  final  tally  reflects  at  least  18  points  for  every 
student.  If  these  are  removed  to  reflect  only  those  points  which  are  earned,  the  final  tally  becomes 
29.7  which  is  the  total  discussed  in  the  introduction. 


-  Whole  Person  Score  (WPS)  =  (Cum  LPs  +Cum  GPA  Pts)/Total  hours  (42) 

Student  B’s  WPS  =  (47.70  +  99.00)742  =  (146.7)742  =  3.49 

-  Had  Student  B  received  an  additional  3.4  GPA  or  Leadership  Points,  he  would  have  been  the  DG,  not 
Student  A. 
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Appendix  B 


Stratified  Systems  Theory  Organizational  Levels  And  Strata 


Strata 

Task  Complexity 

Time  Span  to 
Complete 

Organizational  Domain 

Stratum  VII 

Put  Business  Units 
into  Society 

20-50  years 

Strategic 

Stratum  VI 

World-Wide 

Diagnostic 

Accumulation 

10-20  years 

Strategic 

Stratum  V 

Unified  Whole 

System 

2-5  years 

Strategic 

Stratum  IV 

Parallel  Processing 

2-5  years 

Operation 

Stratum  III 

Alternative 

Pathways 

1-2  years 

Direct 

Stratum  II 

Diagnostic 

Accumulation 

3  months  to  1  year 

Direct 

Stratum  I 

Direct  Judgment 

1  day  to  3  months 

Direct 

Source:  Berlain  Hatfield,  Jr., 
(Maxwell  AFB,  Air  Command 


Strategic  Leadership  Development:  An  Operation  Domain  Application, 
and  Staff  College,  1997),  6. 
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US  Army 


Direct 

Type  of  Influence 

Mostly  Direct 

Size  of  Unit  or 
Organization 

Teams,  Sections, 
Branches,  Small 
Units,  Small  and 
Large  Groups 

Representative  #  of 
Subordinates 

3-600-t 

Time  Span  of  Work 

3-12-1-  Months 

Level  of  Warfighting 

Tactical 

Corresponding  area  of 
Warfighting  Influence 

Roughly  less  than 

5,000-10,000 

meters 

Level  of  Headquarters 

Team,  Squad, 
Section,  Platoon, 
Company, 

Battalion 

Source:  Army  Field  Manual  22-100, 
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Organizational  Strategic 

Direct  and  Indirect  Mostly  Indirect 

Large  Units  and  Mass  Organizations 

Organizations  And  Groups  of 

Organizations 

600-12,000-t-  50, 000-500, 000-t- 

2-10  Years  IO-20-i- Years 

Tactical  Operational  and 

_ Strategic _ 

10-15-1- km  National, 

Sector  and  Continental,  and 

Regional _ Global _ 

Battalion,  Corps,  Numbered 

Directorate,  Army,  Unified  and 

Brigade,  Division  Specified 

Command, 
ARSTAFF, 
MAJCOM,  DOD, 
and  NCA 


ip,”  3-22 


Appendix  D 


AZIMUTH,  Version  I 

Items  Organized  by  the  Leadership  Elements 


Supervisory  Skills. 

Interpersonal  focus 

Backs  his  or  her  subordinates 

Berates  subordinates  who  make  honest  mistakes 

Concerned  with  subordinates’  needs 

Emphasizes  subordinates’  strengths 

Engenders  enthusiasm  in  subordinates 

Harsh  with  subordinates’  errors 

Helps  subordinates  learn  from  their  mistakes 

Moves  quickly  to  confront  problem  subordinates 

Over  supervises  subordinates 

Shows  interest  in  the  professional  growth  of  subordinates 

Supports  subordinates’  decisions 

Treats  subordinates  with  dignity 

Team  focus 

Builds  winning  teams 

Even-handed  in  distributing  workload 

Gets  subordinates  the  resources  they  need  to  do  their  job 

Inspires  subordinates  to  do  their  best 

Provides  challenging  opportunities  for  all  team  members 
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Selects  good  people  in  putting  together  a  team 

Takes  time  to  find  out  what  subordinates  are  doing 

Mission  focus 

Creates  a  supportive  context  within  which  subordinates  can  accomplish  their  work 

Focuses  subordinates’  efforts  to  accomplish  the  mission 

Is  a  good  leader 

Provides  the  coordination  for  subordinate  organizations  to  accomplish  interrelated  tasks 

Tactical  and  Technical  Competence 

Problem  solving  skills 

Can  make  quick  decisions  when  circumstances  call  for  them 

Is  a  quick  study 

Is  quick  to  develop  an  understanding  of  complex  situations 

Jumps  to  conclusions  before  the  facts  are  in 

Makes  good  decisions 

Recognizes  emerging  problems  quickly 

Sees  the  pattern  in  seemingly  unrelated  problems 

Sorts  out  what’s  really  important  from  what  isn’t 

Understands  an  issue  before  making  a  decision. 

Knowledge 

Accomplished  professional 

Fails  to  learn  important  technical  aspects  of  the  business  he  or  she  is  overseeing 

Highly  capable  at  current  job 

Is  behind  the  power  curve  on  key  issues 

Is  technically/tactically  competent 

Knows  policy  or  doctrine 

Lacks  sufficient  technical  competence 

Needs  extensive  guidance 
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Planning/organizational  skills 

Doesn’t  meet  promised  deadlines 

Fails  to  stay  focused  on  primary  issues 

His  or  her  plans  need  frequent  revision 

Is  able  to  envision  several  different  scenarios  when  planning  an  action 

Is  able  to  stay  focused  on  the  primary  effort 

Is  well  organized 

Lacks  long  term  vision 

Sees  the  “big  picture” 

Political  Skills 

Is  interested  in  broad  political  and  societal  issues 

Is  sensitive  to  political  issues  that  may  affect  his  or  her  own  responsibilities 

Seeks  knowledge  about  world  political  and  economic  conditions 

Shows  good  judgment  in  politically  sensitive  matters 

Treats  peers  with  dignity 

Treats  superiors  with  dignity 

Understands  that  politics  are  a  key  part  of  his  or  her  profession 

Ethics 

Abides  by  high  ethical  standards 

Allows  others  to  take  heat  for  his  or  her  own  failures 

Behaves  with  questionable  ethics 

Follows  the  guidelines  he  or  she  sets  for  others 

Honest 

Misuses  subordinates  to  advance  his  or  her  own  career 

Takes  credit  for  other’s  work 
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Takes  special  privileges  for  him-  or  herself 

Tolerates  backstabbing  in  his/her  organization 

Values  his  or  her  own  career  over  the  good  of  the  organization 

Communication/Influence 

Communicates  clearly. 

Establishes  and  uses  informal  communication  networks 

Keeps  subordinates  well  informed 

Listens  when  others  talk 

Persuasive 

Uses  abusive  language 

Uses  praise  to  accomplish  the  organization’s  objectives 

Uses  the  threat  of  punishment  to  accomplish  the  organization’s  objectives 

Social  Maturity 

Has  a  good,  non-hostile  sense  of  humor 

Has  a  sincere  interest  in  what  others  have  to  say 

Is  intolerant  of  criticism 

Is  open  minded 

Is  unwilling  to  admit  to  a  mistake 

Stays  composed  when  under  personal  attack  by  others 

Treats  subordinates  as  valuable  team  members 

Uses  foul  language  excessively 

Self-Centeredness 

Criticizes  subordinates  in  front  of  others 

Has  an  arrogant,  superior  attitude 

Is  aloof,  unapproachable 
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1 .  Is  impressed  with  his  or  her  own  rank  and  status 

Is  vindictive 

Likes  to  draw  attention  to  him-  or  herself 

Often  loses  his  or  her  temper 

Talks  down  to  subordinates 

Thinks  he  or  she  is  right  even  when  others  in  the  know  disagree 

Wants  it  done  “my  way  or  no  way” 

Compulsive  Behavior 

Insists  on  precision  in  trivial  matters 

Is  a  workaholic 

2.  Is  intolerant  of  uncertainty 

Looks  for  the  one  perfect  solution 

Micromanages 

Nit  picks 

Wants  a  great  deal  of  information  before  making  routine  decisions 

Source:  Dr  Stan  Halpin,  The  Leader  AZIMUTH  Check:  A  Leader  Self-Development  Tool,  (Army 
Research  Institute,  Ft  Leavenworth  Field  Office,  1994),  Appendix  A. 
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Appendix  E 


AZIMUTH,  Version  II:  Items  Organized  by  the  Leadership  Elements 
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Appendix  F 


Leader  Azimuth  Check  -  Eigenvalue/Total  Variance 
1999  Army  Command  and  General  Staff  College  Results 


A  Principal  component  analysis  using  the  leader  self-assessment  data  produced  12  components  with 
eigenvlaues  greater  than  one.  These  twelve  components  accounted  for  58%  of  the  variance  (See  Initial 
Eigenvalues  and  Total  Variance  Explained).  A  rotated  component  matrix,  using  varimax  with  Kaiser 
normalization,  converged  after  30  iterations.  Items  which  loaded  0.400  or  greater  on  any  of  the  twelve 
components  are  reported  (See  Analysis  of  Rotated  Component  Matrix). 

Initial  Eigenvalues 

Component  Total  %  of  Variance  Cumulative  % 


1 

23.604 

32.783 

32.783 

2 

3.143 

4.366 

37.149 

3 

2.297 

3.191 

40.340 

4 

2.157 

2.996 

43.336 
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5 


1.816 


2.522 


45.858 


6  1.451  2.015 

7  1.360  1.889 

8  1.290  1.792 

9  1.204  1.673 

10  1.142  1.585 

11  1.099  1.527 

12  1.065  1.479 


47.873 


49.763 


51.554 


53.227 


54.813 


56.339 


57.818 


Total  Variance  Explained 

Rotation  Sums  of  Squared  Loadings 

Component  Total  %  of  Variance 


1 

7.304 

10.144 

10.144 

2 

7.134 

9.908 

20.052 

3 

5.398 

7.497 

27.549 

Cumulative  % 
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4 

3.536 

4.911 

32.460 

5 

3.382 

4.698 

37.158 

6 

3.161 

4.390 

41.548 

7 

2.331 

3.238 

44.786 

8 

2.317 

3.218 

48.004 

9 

2.165 

3.006 

51.010 

10 

1.848 

2.567 

53.577 

11 

1.802 

2.502 

56.080 

12 

1.251 

1.738 

57.818 

Analysis  of  Rotated  Component  Matrix 


Component  1 

Item  Item  Content  (abbreviated) 


Loading 


59  Is  trustworthy.  0.689 

60  Sets  Ethical  example  for  others.  0.612 

47  Support  equal  opportunity  for  all  persons  0.606 

68  Effective  on  the  job.  0.594 

57  Demo’s  morale  courage  (does  what  is  right).  0.585 

39  Does  what’s  necessary  to  complete  mission.  0.580 

(within  ethical  limits) 

37  Completes  assigned  missions  to  standard.  0.575 


46 


48  Creates  climate  of  fairness  in  organization.  0.547 

66  Demo’s  appropriate,  soldier  skills.  0.527 

7 1  This  person  is  a  good  leader.  0.526 

72  Some  one  I  would  follow  into  combat.  0.521 

67  Is  a  clear  thinker.  0.520 

70  Physic  ally  fit  for  the  job.  0 . 504 

53  Place  organization  before  own  personal  gain.  0.466 
50  Treats  others  with  respect.  0.466 

Component  2 

Item  Item  Content  (abbreviated) 


32  Anticipates  how  different  plans  will  look.  0.670 

41  Refines  plans  to  exploit  unforeseen  opportunities.  0.666 

33  Develops  effective  plans  to  ach.  org.  goals.  0.643 

8  Generates  innovative  solutions  to  unique  problems  0.596 

40  Monitors  execution  of  plans  to  ID  problems.  0.594 

29  Helps  org.  adapt  to  changing  circumstances.  0.569 

10  Makes  sound  decisions  in  a  timely  manner.  0.477 

18  Is  an  effective  teacher.  0.473 

67  Is  a  clear  thinker.  0.470 

35  Sets  clear  priorities.  0.465 

28  Encourages  open  discussion  to  improve  org.  0.459 

25  Focuses  the  org.  on  mission  accomplishment.  0.45 1 

31  Willingly  accepts  new  challenges.  0.437 


Loading 
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72  This  person  is  someone  I  would  follow  in  combat  0.429 


42  Accurately  assesses  org.  strengths.  0.425 

7 1  This  person  is  a  good  leader.  0.414 

68  Is  effective  on  the  job.  0.407 

43  Accurately  assesses  org.  weaknesses.  0.405 

16  Sets  clear  performance  expectations.  0.403 


Component  3 

Item  Item  Content  (abbreviated) 


15  Acknowledges  good  performance  of  others.  0.657 

12  Creates  supportive  work  environment.  0.629 

26  Treats  others  as  valuable  team  members.  0.596 

14  Inspires  people  to  do  their  best.  0.547 

48  Creates  a  climate  of  fairness  in  the  org.  0.542 

50  Treats  others  with  respect.  0.507 

47  Actively  supports  equal  opportunity  for  all.  0.452 

23  Encourages  cooperation  among  team  members.  0.449 
46  Takes  time  to  find  out  what  subord.’s  are  doing  0.442 
28  Encourages  open  discussion  to  improve  org.  0.428 

69  Maintains  effective  interpersonal  relations.  0.423 

52  Considers  needs  of  own  &  others’  family  members.  0.407 
20  Provides  opportunities  to  learn.  0.403 


Loading 
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Component  4 

Item  Item  Content  (abbreviated) 


9  Ignores  information  that  conflicts  with  own.  0.616 

36  Unwilling  to  modify  plan  w/circumstances  change.  0.559 
21  Seldom  delegates  authority.  0.547 

34  Leaves  key  events  to  chance.  0.465 

44  Makes  org.  changes  for  no  apparent  reason.  0.463 

5 1  Claims  credit  for  others’  work.  0.444 

27  Becomes  defensive  when  given  critical  feedback.  0.422 


Component  5 

Item  Item  Content  (abbreviated) 


64  Possesses  an  even  temperament.  0.783 

61  Does  not  display  extreme  anger.  0.725 

62  Exhibits  wide  mood  swings.  0.662 

63  Maintains  calm  disposition  under  stress.  0.660 

65  Seems  to  behave  unpredictably.  0.452 


27  Becomes  defensive  when  given  critical  feedback.  0.407 


Loading 


Loading 
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Component  6 

Item  Item  Content  (abbreviated)  Loading 


54  Takes  advantage  of  others  to  advance  own  career.  0.675 


55  Takes  privileges  not  allowed  others.  0.629 

56  Behaves  with  questionable  ethics.  0.577 

49  Excludes  some  from  team  activities.  0.509 

5 1  Claims  credit  for  others’  work.  0.484 

65  Seems  to  behave  unpredictably.  0.423 

44  Makes  org.  changes  for  no  apparent  reason.  0.402 


Component  7 

Item  Item  Content  (abbreviated) 

Loading 

2  Explains  own  ideas  so  they  are  understood. 

0.712 

3  Keeps  others  well  informed. 

0.691 

4  Eistens  well. 

0.561 

Component  8 

Item  Item  Content  (abbreviated) 

Loading 

22  Actively  participates  in  org.  activities. 

0.803 

24  Encourages  org.  activities. 

0.735 

23  Encourages  cooperation  among  team  members. 

0.473 
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Component  9 

Item  Item  Content  (abbreviated) 


Loading 


45  Rarely  conducts  after-action  reviews.  0.616 

19  Often  uses  counseling  to  provide  perf.  feedback.  0.553 
16  Sets  clear  performance  expectations.  0.405 


Component  10 

Item  Item  Content  (abbreviated) 


Loading 


43  Accurately  assesses  org.  strengths.  0.721 

42  Accurately  assesses  org.  weaknesses.  0.699 


Component  11 

Item  Item  Content  (abbreviated) 


Loading 


6  Writes  poorly.  0.695 

1  Does  not  provide  clear  direction.  0.554 


Component  12 

Item  Item  Content  (abbreviated) 


Loading 
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58  Is  not  sensitive  to  ethical  impacts  of  decisions. 


0.608 
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Appendix  G 


Leader  Azimuth  Check  -  Reliability  Analysis 
1999  Army  Command  and  General  Staff  College  Results 

Reliability  Analysis  (coefficient  alpha)  for  Leader  Azimuth  Check,  Version  2 


Scale 


Alpha 


Communicating  0.6696 


Decision-Making  0.7642 


Motivating 


0.7927 


Developing 


0.7284 


Building 


0.7628 


Learning 


0.7084 
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Planning  and  Organizing 


0.8637 


Executing  0.7173 

Assessing  0.6793 

Respect  0.8413 

Selfless  Service  0.6569 

Integrity  0.7773 

Emotional  Stability  0.8693 
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Appendix  H 


Interview  Request  Staff  Summary  Sheet 
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SIAFF  SUMMARY  SHEET 


TO 

*tTIOh 

if  M'w  Aflfue.fUdt 

TO 

ACTWtS 

1  twivf  wmnE 

ACSC.- 

Appr 

*/' m  X  fV\~ 

aCSC^ 

Appr 

S/- 

DPcn 

i  J  3^m 

DEA 

'ACSC.' 

ACPf 

ACSC^ 

Aptif 

TUT^jm^zru^ 

CUTS 

m 

PFB 

f  Sr't 

.  ACSC.- 

Appr 

ACSC^ 

Appr 

'  DPD3 

DEC 

■n 

.ACSC^ 

ACSCi' 

Appr 

jL7  3lkc.£  ^ 

P?|M 

DE 

ACSC.- 

Appr 

ACKt 

Appr 

MD 

-j 

CC 

iL#kJUriV  UnMUKffiiHQ  EIMI 

5r»Ci 

rwwrt 

n^nauic 

Mai  Meeker 

ACSC.iScmff?  13-671* 

Tnam 

22  Feb  W 


itiiKT 

Leadership  Ajjessmctii  ai  ACSCi'Ttcseareh  Paper  _ 

amttit 

L.  I'be  letLer  aL  lab  1  miucFla  intervLews  with  eacli  afihe  afgaiuzalimnsi  listed  abc^vt  The  intent  is  Id  gather 
[injlirtiinaiy  4aU  ttorrt  sirtkir  Innkra  i»iihin  AC5C  on  what  leidcrslii|i  skiLla,  bchaviofi,  <>r  auribyws  thc^ 
consider  impofiaLit  for  ACSC  £taduai£s. 

2.  Tlie  inierviei^fi  '*jll  7Ufi|>c:qt  ACSC  rescaruh  project  aftemptiiig  to  define  cihjective  criien!!  iis  ii  basis 
for  leadership  poiM  assessmcni  ai  ACSC-  The  iiHeiview^s  stuniid  tahe  tio  langcT  than  20-  minutes  and  vvill  be 
confined  to  Ihe  seyen  tituestiana  lisicd  in  the  leicer  at  tah  I 

J,  ftEC:OMMENL>ATlON;  Individuals  lislcd  above  ajjee  to  the  interviews  by  sieving  the  52iS 
ci'Hsrdinatioii  Wnci!  al?i>ve.  Interviews  witl  be  at  the  mdivkhials  convience  and  wilfbe  ertinJiriHed  by 
Maj  Meeker. 

DAVID  A.  MILEWSKI,  Li  Col.  USAF  I  Tab 

D^T¥tK!T^  livalualion  Division  Imerview  Rciiuesl  Letter 


tF  Fanuuit.  SEP  BUflf  VA} 


Pf  MUEHmSI  K  LSm. 
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DEf  AI7TMENT  CF  THE  Alfi  FORCE 

Am-  u>HiVEnHrr  iaeihci 


MEMORANDUM  FOR  ACSCrtl’VV 


FROM:  ACSC.'Sem  D? 

SUBJECT:  Reqiwsi;  f«r  Jnler^i^w 

t.  Accivaftc  liCBidtmhip  oppriiiisal  in  an  educatLoiul  mvicioiiiiKfiC  lequim  b[>lti  pnx^ss  Dud  ‘criteria 
guidaiKe  for  all  raters.  ACSC'a  currojit  program,  centering  around  IcadcratiLp  poiota,  has  detailed 
prueesa  in  place,  boi  biiJcN.H»b;itK:Live  crlLeira  j^uldane-t:.  Tti  |hL$,  I’ni  Di.Wwiirg 

research  boohing  at  (hK  fmsibililyoFckvelDpiTi]^  n  model  IbctBod  on  llie  specific  skills  and 

bcbia^DFS  CHpeded  Tnur  AC5C  giaduates. 

2.  Tlic  research  fticiiseg  on  three  questions:  I)  l^Tiat  Leadership  skills  arc  needed  in  ACSC  graduaiea, 
2}  Wks  aj  ACSC  earn  uhsti'irt  ihifsii  skill^i,  and  3j  i™ig>i(itijj[  shuald  k  jfiven  lo  (he  iitsights  (ran 
various  otrservers?  Because  the  tbeus  is  on  A'CSC  graduades  only,  I'm  requesting  sn  opportuniiy  io 
interview  mcnibcis  of  the  ACSC  senior  Icadcislup.  The  hope  is  their  espccicncc  and  perspective  can 
provide  a  starting  poittt  for  delineating  s^iocifk  Ittadcrdtip  skills  that  can  he  efTectlvely  observed  in  nn 
educaLinnal  .scLting.  The  interview  shnuld  Lake  ran  li.]n|[er  ihan  211  minutes  mid  w]||  be  limited  (0  (he 
fdlbowin*  qucsiinnsr 

R.  WhiM  should  tte  (he  purpose  of  (he  current  leadership  odsesirrcrK  program  in  place  a(  ACSC? 
{exsonplcs:  oieasore  of  pcrfotmance  only,  possible  indicator  of  success  in  firtaue  jobs,  arc.) 

h.  Based  nn  yntw  anitt'cr  tn  die  pr^vioas  questi^m,  what  eweral  I  typw  of  ski  lls,.  behllvio^s^  or 
Uaits  sk>uld  form  the  basis  for  nsscssmeni? 

e.  Looking  at  (he  Army's  [.^ader  Azimuth  Check  (provided  during  the  intcfvtewk  wfiich  i(ants 
effectively  describe  those  skills  or  behaviem  you  ntentionod  earlier?  Are  ihcie  any  items  yw 
would  add  or  delete? 

d.  Fman  ycHir  petspeclite  {Cimnmandarti,  etc-)  are  there  any  of  your  listed  skills  or 
behaviors  thm  contribute  more  (o  a  student's  success  (ban  others  (initiative  over  v.ell- 
orgrmiited  for  estample.) 

e.  ^IsD  do  yvu  believe  could  coirPibirte  accurate  feedback  cm  dinae  trait.';,  skills,  <ir  hehaviois 
listed  earlier?  (instructnra^,  peers,  ups  iifneers,  smiur  staff) 

f.  Are  tiny  of  (hose  individuals  in  a  better  position  to  observe  a  student's  abilities  than  the  others 
mentioned?  (ex.  course  inslructors  oa'ci  peers) 

g.  If&a^  is  the  adtfaulflgt;  great  annugb  in  vneighl  ihciT  sewe  more?  lA'hat  wieigfiiing  wiauld  you 
give  them? 

3.  If  you  have  any  questions,  please  contact  me  via  email  al  Manha.Meckcr-^MBXwell.AF.MlI.. 

MARTHA  A.  MLLKLK,  Maj,  USAF 
Seminal  07 
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Appendix  I 


Research  Question:  What  to  Measure 

“What  skills,  behaviors,  or  traits  in  general  should  form  the  basis  for 

leadership  assessment?” 


Seminar 

Leaders 

First  Semester 

Seminar  Leaders 

Second  Semester 

Commandant 

Operations  Officers 

Academics: 

DE/DEA/DEB/DEC 

Communication 

Class  participation  such  as 

participating  in  group  discussions  and 

attending  seminar  functions 

Very  situational 

Team  Work  including 

interacting  well,  supporting,  ability 

to  follow 

Team  Work 

Taking  on  leadership  responsibility 

such  as  sponsoring  special  projects  at 

ACSC 

Level  headed,  Calm  under  stress 

Willing  to  make  the  hard 

decision  to  do  what’s  right 

Positive  attitude  in  both 

what  a  seminar  leader  does  and 

say 

Team  Work:  Contributing  to  the 

overall  team  goals  and  efforts 

Able  to  articulate  ideas 

Ability  to  inspire  confidence 

Organization  Skills:  Time 

management  and  keeping  the  team 

on  course 

Facilitating  group  dynamics  and 

providing  a  good  work  environment 

Able  to  see  several  sides  of  a 

situation 

Respect 

Positive  attitude 

Respect  for  others 

Able  to  assess  individual 

strengths  within  a  group  in  order  to 

effectively  utilize  them 

Vision 

Ability  to  get  the  group  to 

perform  mission 

Ability  to  mesh  a  team  and  to  get 

participation  by  all  of  its  members 

Delegation 

Ability  to  inspire  trust 

Team  building: 

encouraging,  directing,  and 

inspiring 

Initiative 

Able  to  motivate  individuals 

to  accomplish  the  given  task 

Team  Work 
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Selflessness 


Team  work  or  activities 


Ability  to  keep  morale  up  in  a  unit 


EthicsA^  alues/Character 


Communication  skills: 


Likes  all  items  but  would  focus  on 


Communication 


Ability  get  the  mission  Team  Work 


accomplished 


Risk  taking:  Stepping  out  to  Communication:  One-on-one 


Interpersonal  and  in  a  group 


build  a  program  for  example 


and  in  a  group,  written  and  verbal 


Contributes 


Decision  Making 


Creativity:  Need  to  see  Willing  to  take  the  responsibility 


esprit/performance 


different  ways  to  approach  a  problem  to  lead 


Critical  Analysis 


Creative  Thinking 


OralAVritten  Communication 


Ability  to  work  with  Joint  and 


Motivating:  Inspiring  people  Team  Building 
to  pitch  in  with  the  project 


Assessing:  Need  to  see  the  Willing  to  make  a  decision 

problem  not  the  symptoms 


Decision  making  with  Able  to  accept  criticism 

incomplete  information 


Ability  to  accomplish  the  set  Able  to  provide  criticism  in  an 

goals.  Just  volunteering  for  community  appropriate  manner 
service  won’t  make  it  as  that’s  not  tied 
to  taking  the  unit  forward. 


Knowledge  of  Job  and  People  Able  to  get  along  with  others 


Combined  Forces  including 
cultural/social  sensitivity 


regardless  of  personal  likes/Group 
dynamics  are  important 
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Appendix  J 


Research  Question:  Leader  Azimuth  Check 
“Looking  at  the  Army’s  Leader  Azimuth  Check,  which  items 
effectively  describe  those  skills  or  behaviors  you  mentioned 
earlier?  Are  there  any  items  you  would  add  or  remove?” 


Seminar  Leaders 

First  Semester 

Seminar  Leaders 

Second  Semester 

Commandant 

Operations  Officers 

Academics: 

DE/DEA/DEB/DEC 

Communication 

Communication 

Exclude  Planning  and  Organization 

Team  Building 

Agrees  with  all  the  items  but  especially  likes 

Integrity 

Communicating 

Need  to  add  “trust”  item  as  this  is 
the  heart  and  soul  of  leading 

Communication 

Execution 

Working  as  a  team  leads  to 
“Motivating”  and  “Building” 

Communication 

Executing 

Decision  Making 

Emotional  Stability 

Decision  Making 

Motivating 

Motivating 

Decision  Making 

Motivating 

Planning  and  Organization 

Communication 

Communication 

Building 

Big  Four  would  be 

Motivation:  Inspire  and  encourage 

Motivating 

Integrity 

Communication 

Decision  Making 

Developing 

Selfless  Service 

Decision  Making 

Respect:  Paramount 

Team  Building 

Emotional  Stability 

Motivating 

Integrity:  Number  I ,  speaks  to 
credibility  as  a  leader 

Respect:  Must  provide  an  environment 
where  individuals  are  capable  of 
expressing  views 

Emotional  Stability 

Integrity 

Disagree  with  Learning:  Also 
“Defensiveness”  is  more  about 
attitude/self  esteem;  if  included,  place 
in  different  category 

Selfless  Service 

Communication 

Would  also  include 

Add  category  for  “Open  Minded”  or 
“Hexible” 

Keep  all  items  but  realize  much  of  the 
Planning  and  Organization  is  already 
done  for  a  student 

Decision  Making 

Learning 

Communication 

Communication 

Building 

Planning/Organization 

Motivating 

Motivating:  Best  program  is  worthless 
unless  you  can  move  people  to  take  it 
forward 

Integrity 

Assessing 

Building 

Assessing 

Disagrees  with  “leaving  key 
events  to  chance”  in 
Planning/Organization 
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Would  look  at  adding  initiative 


Planning  and  Organization 


Relook  Emotional  Stability  as  sometimes  Executing 
commanders  must  show  emotion 


Include  all  Elements  Respect 


Decision  Making 


Selfless  Service 


Integrity 


Motivating 


Disagrees  with  “does  whatever  is 
necessary  within  ethical  limits  to 
accomplish  mission”  Already 
trying  to  do  too  much  with  too 
little 


Take  out  Others/Emotional 
Stability/Planning 


Communication 


Decision  Making 


Communication 


Decision  Making 


Motivating 


Developing 


Building 


Learning 


Planning/Organization 


Executing 


Assessing 


Respect 


Selfless  Service 


Integrity 


Emotional  Stability 


Other 


Selfless  Service 


Integrity 


Emotional  Stability 


Others 


Building 


Communication 


Decision  Making 


Integrity 


Emotional  Stability 


Building 


Learning 


All  are  good  but  big  three  are 


Respect 


Communication 


Decision  Making 


Motivating 


Developing 


Building 


Learning 


Executing 


Assessing 


Respect 


Selfless  Service 


Integrity 


Note:  Indented  blocks  are  considered  part  of  the  parent  comment  above  it 


Seminar  Leaders  Comm/Operations  Dean/Faculty  Total 


(n=9) 


8 


5 


6 


1 


3 


2 


(n=5) 


(n=4) 


3 


3 


2 


1 


(n=18) 


15 


11 


12 


3 


10 


Azimuth  Elements 


Communication 


Decision  Making 


Motivating 


Developing 


Building 


Learning 


Planning  and  Organization 


Executing 


Assessing 


Respect 


Selfless  Service 


Integrity 


Emotional  Stability 


Other 


Note:  Positive  and  Negative  statements  are  combined  in  the  resulting  totals.  Example: 
two  members  of  the  academic  staff  voted  for  “Emotional  Stability”  while  one  voted 
against.  Einal  total  is  reflected  as  one.  The  lowest  tally  for  any  element  is  zero. 
Negative  values  are  not  reflected. 


1 

1 

0 

0 

0 

1 

1 

1 

2 

1 

2 

2 

1 

3 

2 

1 

0 

0 
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Appendix  K 


Research  Question:  Leader  Azimuth  Check 
“Looking  at  the  72  items  contained  in  the  Leader  Azimuth  Check,  which 
are  observable,  somewhat  observable,  or  not  observable?” 


Leader  Azimuth  Check 


Seminar  Leaders 

Ohservahle 

Somewhat 

Observable 

Not 

Observable 

Communication 

1 .  Does  not  provide  clear  direction 

bcdeghi 

af 

2.  Explains  own  ideas  so  they  are  easily  understood 

abcdefghi 

3.  Keeps  others  well  informed 

adegi 

bch 

f 

4.  Listens  well 

abcdefgi 

h 

5.  Tells  it  like  it  is 

acdfhi 

beg 

6.  Writes  poorly 

abceghi 

df 

Decision  Making 

7.  Delays  decisions  unnecessarily 

ai 

bcdegh 

f 

8.  Generates  innovative  solutions  to  unique  problems 

abceghi 

df 

9.  Ignores  information  that  conflicts  with  own  assumptions 

ahi 

bcdefg 

10.  Makes  sound  decisions  in  a  timely  manner 

acei 

bdfgh 

1 1 .  Willing  to  revisit  a  decision  when  new  information  calls 
for  it 

aghi 

bcdef 

Motivating 

12.  Creates  a  supportive  work  environment 

abcdefghi 

13.  Disciplines  in  a  firm,  fair,  and  consistent  manner 

cei 

abdfgh 

14.  Inspires  people  to  do  their  best 

abcdeghi 

f 

15.  Often  acknowledges  good  performance  of  others 

abcefghi 

d 

16.  Sets  clear  performance  expectations 

acegi 

h 

bdf 

Developing 

17.  Does  not  encourage  professional  growth 

i 

abde 

chfg 

18.  Is  an  effective  teacher 

ai 

bdfgh 

ce 

19.  Often  uses  counseling  to  provide  performance  feedback 

i 

eg 

abcdfh 

20.  Provides  opportunities  to  learn 

ei 

bdfg 

ach 

21.  Seldom  delegates  authority 

egi 

ac 

bdhf 

Building 

22.  Actively  participates  in  organizational/unit  activities 

abcdefghi 

23.  Encourages  cooperation  among  team  members 

abcdefghi 

24.  Encourages  organizational/unit  activities 

abcdefghi 

25.  Eocuses  the  organization/unit  on  mission  accomplishment 

abcdgi 

efh 
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26.  Treats  others  as  valuable  team  members 

abcdefghi 

Learning 

27.  Becomes  defensive  when  given  critical  feedback 

aceghi 

bdf 

28.  Encourages  open  discussion  to  improve  the 
organization/unit 

acdei 

bgh 

f 

29.  Helps  organization/unit  adapt  to  changing  circumstances 

abei 

cdfgh 

30.  Seems  to  be  realistic  about  own  personal  limitations 

acdi 

befgh 

3 1 .  Willingly  accepts  new  challenges 

acdeghi 

bf 

Planning  and  Organizing 

32.  Anticipates  how  different  plans  will  look  when  executed 

i 

bcdeh 

afg 

33.  Develops  effective  plans  to  achieve  organizational  goals 

egi 

bcdh 

af 

34.  Leaves  key  events  to  chance 

ei 

cdgh 

abf 

35.  Sets  clear  priorities 

aceghi 

bdf 

36.  Unwilling  to  modify  original  plan  when  circumstances 
change 

ceghi 

bd 

af 

Observable 

Somewhat 

Observable 

Not 

Applicable 

Executing 

37.  Completes  assigned  missions  to  standard 

acegi 

bdfh 

38.  Does  not  meet  mission  timeliness 

aceghi 

bdf 

39.  Does  whatever  is  necessary  (within  ethical  limits)  to 
complete  the  mission 

cefi 

abdgh 

40.  Monitors  execution  of  plans  to  identify  problems 

i 

bcegh 

adf 

41.  Refines  plans  to  exploit  unforeseen  opportunities 

i 

bceh 

adfg 

Assessing 

42.  Accurately  assesses  the  organization/unit’s  strengths 

i 

begh 

aedf 

43.  Accurately  assesses  the  organization/unit’s  weaknesses 

i 

begh 

aedf 

44.  Makes  organizational  changes  for  no  apparent  reason 

i 

abcdefgh 

45.  Rarely  conducts  after-action  reviews 

i 

abcdefgh 

46.  Takes  time  to  find  out  what  subordinates  are  doing 

ehi 

c 

abdfg 

Respect 

47.  Actively  supports  equal  opportunity  for  all  persons 

acdefgi 

h 

b 

48.  Creates  a  climate  of  fairness  in  the  organization/unit 

acegi 

bdfh 

49.  Excludes  some  from  team  activities 

acdefghi 

b 

50.  Treats  others  with  respect 

abcdefghi 

Selfless  Service 

5 1 .  Claims  credit  for  other’ s  work 

abcdeghi 

f 

52.  Considers  the  needs  of  own  and  others’family  members 

acehi 

bfgd 

53.  Places  the  welfare  of  the  organization  before  own 
personal  gain 

acdegi 

hf 

b 

54.  Takes  advantage  of  others  to  advance  own  career 

adegi 

bcfh 

55.  Takes  privileges  not  allowed  others 

aei 

bed 

fgh 

Integrity 

56.  Behaves  with  questionable  ethics 

abcdegi 

fh 

57.  Demonstrates  moral  courage  (does  what  is  right) 

acdegi 

bfh 

58.  Is  not  sensitive  to  the  ethical  impacts  of  decisions 

acdegi 

bh 

f 

59.  Is  trustworthy 

acdegi 

bfh 

60.  Sets  the  proper  ethical  example  for  others 

acdegi 

bfh 

Emotional  Stability 

61.  Does  not  display  extreme  anger 

acdefghi 

b 

62.  Exhibits  wide  mood  swings 

abcdefghi 

63 


63.  Maintains  calm  disposition  under  stress 

abceghi 

df 

64.  Possesses  an  even  temperament 

abcdefghi 

65.  Seems  to  behave  unpredictably 

abcdefghi 

Other 

66.  Demonstrates  appropriate  warrior  skills 

adgi 

ch 

bef 

67.  Is  a  clear  thinker 

abcdegi 

fh 

68.  Is  effective  of  the  job 

acdegi 

bh 

f 

69.  Maintains  effective  interpersonal  relations  with  others 

abcdefghi 

70.  Physically  fit  for  the  job 

acdeghi 

bf 

7 1 .  This  person  is  a  good  leader 

acdegi 

bh 

f 

72.  This  person  is  someone  I  would  follow  into  combat. 

cdehgi 

ab 

f 

Note:  Each  letter  represents  an  individual  subject’s  response 


Leader  Azimuth  Check 


Commandant/Operations  Officers 

Observable 

Somewhat 

Observable 

Not 

Observable 

Communication 

66.  Does  not  provide  clear  direction 

ADE 

c 

B 

67.  Explains  own  ideas  so  they  are  easily  understood 

BODE 

A 

68.  Keeps  others  well  informed 

AD 

BCE 

69.  Listens  well 

BODE 

A 

70.  Tells  it  like  it  is 

BODE 

A 

71.  Writes  poorly 

ABODE 

Decision  Making 

72.  Delays  decisions  unnecessarily 

A 

BDE 

C 

73.  Generates  innovative  solutions  to  unique  problems 

CAD 

BE 

74.  Ignores  information  that  conflicts  with  own  assumptions 

C 

BDE 

A 

75.  Makes  sound  decisions  in  a  timely  manner 

ABD 

CE 

76.  Willing  to  revisit  a  decision  when  new  information  calls 
for  it 

D 

ABCE 

Motivating 

77.  Creates  a  supportive  work  environment 

ABD 

CE 

78.  Disciplines  in  a  firm,  fair,  and  consistent  manner 

D 

A 

BCE 

79.  Inspires  people  to  do  their  best 

BD 

ACE 

80.  Often  acknowledges  good  performance  of  others 

ABCDE 

8 1 .  Sets  clear  performance  expectations 

D 

AB 

CE 

Developing 

82.  Does  not  encourage  professional  growth 

ACDE 

B 

83.  Is  an  effective  teacher 

D 

B 

ACE 

84.  Often  uses  counseling  to  provide  performance  feedback 

D 

B 

ACE 

85.  Provides  opportunities  to  learn 

BD 

C 

AE 

86.  Seldom  delegates  authority 

AD 

B 

CE 

Building 

87.  Actively  participates  in  organizational/unit  activities 

ABCDE 

88.  Encourages  cooperation  among  team  members 

BODE 

A 

89.  Encourages  organizational/unit  activities 

BODE 

A 

90.  Eocuses  the  organization/unit  on  mission  accomplishment 

BD 

ACE 

91.  Treats  others  as  valuable  team  members 

BODE 

A 

Learning 

92.  Becomes  defensive  when  given  critical  feedback 

ABCDE 
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93.  Encourages  open  discussion  to  improve  the 
organization/unit 

ABCD 

E 

94.  Helps  organization/unit  adapt  to  changing  circumstances 

ACD 

BE 

95.  Seems  to  be  realistic  about  own  personal  limitations 

BD 

ACE 

96.  Willingly  accepts  new  challenges 

ABDE 

C 

Planning  and  Organizing 

97.  Anticipates  how  different  plans  will  look  when  executed 

B 

ACD 

E 

98.  Develops  effective  plans  to  achieve  organizational  goals 

BD 

ACE 

99.  Leaves  key  events  to  chance 

ACDE 

B 

100.  Sets  clear  priorities 

ABDE 

C 

101. Unwilling  to  modify  original  plan  when  circumstances 
change 

DE 

AC 

B 

Observable 

Somewhat 

Observable 

Not 

Applicable 

Executing 

102. Completes  assigned  missions  to  standard 

ABCDE 

103. Does  not  meet  mission  timeliness 

ACDE 

B 

104. Does  whatever  is  necessary  (within  ethical  limits)  to 
complete  the  mission 

BDE 

AC 

105. Monitors  execution  of  plans  to  identify  problems 

BD 

ACE 

106. Refines  plans  to  exploit  unforeseen  opportunities 

B 

ACDE 

Assessing 

107. Accurately  assesses  the  organization/unit’s  strengths 

BD 

ACE 

108. Accurately  assesses  the  organization/unit’s  weaknesses 

BD 

ACE 

109. Makes  organizational  changes  for  no  apparent  reason 

ADE 

BC 

1 10. Rarely  conducts  after-action  reviews 

D 

A 

BCE 

1 1 1. Takes  time  to  find  out  what  subordinates  are  doing 

D 

AB 

CE 

Respect 

1 12.  Actively  supports  equal  opportunity  for  all  persons 

BDE 

AC 

1 13. Creates  a  climate  of  fairness  in  the  organization/unit 

BDE 

AC 

1 14. Excludes  some  from  team  activities 

AD 

BCE 

1 15. Treats  others  with  respect 

ABCDE 

Selfless  Service 

1 16. Claims  credit  for  other’s  work 

AD 

BCE 

1 17.  Considers  the  needs  of  own  and  others ’family  members 

D 

ABEC 

1 18. Places  the  welfare  of  the  organization  before  own 
personal  gain 

AD 

BE 

C 

1 19. Takes  advantage  of  others  to  advance  own  career 

D 

ABCE 

120. Takes  privileges  not  allowed  others 

D 

ACE 

B 

Integrity 

121. Behaves  with  questionable  ethics 

ACD 

E 

B 

122. Demonstrates  moral  courage  (does  what  is  right) 

ABD 

CE 

123. Is  not  sensitive  to  the  ethical  impacts  of  decisions 

D 

ACE 

B 

124. Is  trustworthy 

ABDE 

C 

125. Sets  the  proper  ethical  example  for  others 

BD 

ACE 

Emotional  Stability 

126. Does  not  display  extreme  anger 

ABCDE 

127. Exhibits  wide  mood  swings 

ACDE 

B 

128. Maintains  calm  disposition  under  stress 

ABCDE 

129. Possesses  an  even  temperament 

ABCDE 

130.  Seems  to  behave  unpredictably 

ACDE 

B 
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Other 

68.  Demonstrates  appropriate  warrior  skills 

BDE 

ACE 

69.  Is  a  clear  thinker 

ABCDE 

73.  Is  effective  of  the  job 

ABODE 

74.  Maintains  effective  interpersonal  relations  with  others 

ABCDE 

75.  Physically  fit  for  the  job 

BCDE 

A 

76.  This  person  is  a  good  leader 

ABCD 

E 

77.  This  person  is  someone  I  would  follow  into  combat. 

BCD 

AE 

Note:  Each  letter  represents  an  individual  subject’s  response 


Leader  Azimuth  Check 


Academics  (DE/DEA/DEB/DEC) 

Ohservahle 

Somewhat 

Ohservahle 

Not 

Ohservahle 

Communication 

131. Does  not  provide  clear  direction 

23 

14 

132. Explains  own  ideas  so  they  are  easily  understood 

23 

14 

133. Keeps  others  well  informed 

24 

1 

3 

134. Listens  well 

234 

1 

135. Tells  it  like  it  is 

24 

13 

136. Writes  poorly 

1234 

Decision  Making 

137. Delays  decisions  unnecessarily 

2 

134 

138. Generates  innovative  solutions  to  unique  problems 

23 

14 

139. Ignores  information  that  conflicts  with  own  assumptions 

2 

134 

140. Makes  sound  decisions  in  a  timely  manner 

23 

14 

141.  Willing  to  revisit  a  decision  when  new  information  calls 
for  it 

12 

34 

Motivating 

142. Creates  a  supportive  work  environment 

4 

13 

2 

143. Disciplines  in  a  firm,  fair,  and  consistent  manner 

1 

234 

144. Inspires  people  to  do  their  best 

234 

1 

145. Often  acknowledges  good  performance  of  others 

12 

34 

146. Sets  clear  performance  expectations 

1 

4 

23 

Developing 

147. Does  not  encourage  professional  growth 

14 

23 

148. Is  an  effective  teacher 

24 

13 

149. Often  uses  counseling  to  provide  performance  feedback 

1 

234 

150. Provides  opportunities  to  learn 

14 

23 

151.  Seldom  delegates  authority 

1 

4 

23 

Building 

152.  Actively  participates  in  organizational/unit  activities 

1234 

153. Encourages  cooperation  among  team  members 

234 

1 

154. Encourages  organizational/unit  activities 

1234 

155.Eocuses  the  organization/unit  on  mission  accomplishment 

234 

1 

156. Treats  others  as  valuable  team  members 

1234 

Learning 

157. Becomes  defensive  when  given  critical  feedback 

1234 

158. Encourages  open  discussion  to  improve  the 
organization/unit 

1234 

159. Helps  organization/unit  adapt  to  changing  circumstances 

234 

1 

160.  Seems  to  be  realistic  about  own  personal  limitations 

234 

1 
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161.  Willingly  accepts  new  challenges 

234 

1 

Planning  and  Organizing 

162.  Anticipates  how  different  plans  will  look  when  executed 

2 

1 

34 

163. Develops  effective  plans  to  achieve  organizational  goals 

24 

13 

164. Leaves  key  events  to  chance 

24 

1 

3 

165. Sets  clear  priorities 

1234 

166. Unwilling  to  modify  original  plan  when  circumstances 
change 

1234 

Observable 

Somewhat 

Observable 

Not 

Applicable 

Executing 

167. Completes  assigned  missions  to  standard 

1234 

168. Does  not  meet  mission  timeliness 

1234 

169. Does  whatever  is  necessary  (within  ethical  limits)  to 
complete  the  mission 

234 

1 

170. Monitors  execution  of  plans  to  identify  problems 

234 

1 

171. Refines  plans  to  exploit  unforeseen  opportunities 

1234 

Assessing 

172. Accurately  assesses  the  organization/unit’s  strengths 

4 

13 

2 

173. Accurately  assesses  the  organization/unit’s  weaknesses 

4 

13 

2 

174. Makes  organizational  changes  for  no  apparent  reason 

14 

23 

175. Rarely  conducts  after-action  reviews 

1 

4 

23 

176. Takes  time  to  find  out  what  subordinates  are  doing 

4 

13 

2 

Respect 

177.  Actively  supports  equal  opportunity  for  all  persons 

234 

1 

178. Creates  a  climate  of  fairness  in  the  organization/unit 

234 

1 

179. Excludes  some  from  team  activities 

234 

1 

180. Treats  others  with  respect 

1234 

Selfless  Service 

181. Claims  credit  for  other’s  work 

1234 

182. Considers  the  needs  of  own  and  others’family  members 

234 

1 

18 3. Places  the  welfare  of  the  organization  before  own 
personal  gain 

234 

1 

184. Takes  advantage  of  others  to  advance  own  career 

234 

1 

185. Takes  privileges  not  allowed  others 

1234 

Integrity 

186. Behaves  with  questionable  ethics 

1234 

187. Demonstrates  moral  courage  (does  what  is  right) 

234 

1 

1 88.1s  not  sensitive  to  the  ethical  impacts  of  decisions 

234 

1 

1 89.1s  trustworthy 

234 

1 

190.  Sets  the  proper  ethical  example  for  others 

1234 

Emotional  Stability 

191. Does  not  display  extreme  anger 

1234 

192. Exhibits  wide  mood  swings 

1234 

193. Maintains  calm  disposition  under  stress 

1234 

194. Possesses  an  even  temperament 

1234 

195. Seems  to  behave  unpredictably 

1234 

Other 

70.  Demonstrates  appropriate  warrior  skills 

123 

4 

71.  Is  a  clear  thinker 

234 

1 

78.  Is  effective  of  the  job 

234 

1 

67 


79.  Maintains  effective  interpersonal  relations  with  others 

234 

1 

80.  Physically  fit  for  the  job 

234 

1 

8 1 .  This  person  is  a  good  leader 

1234 

82.  This  person  is  someone  I  would  follow  into  combat. 

234 

1 

Note:  Each  number  (1-4)  represents  an  individual  subject’s  response 


Leader  Azimuth  Check 


Overall 

Observable 

Somewhat 

Observable 

Not 

Observable 

Communication 

196. Does  not  provide  clear  direction 

12 

5 

1 

197. Explains  own  ideas  so  they  are  easily  understood 

15 

3 

0 

198. Keeps  others  well  informed 

9 

7 

2 

199. Listens  well 

15 

3 

0 

200. Tells  it  like  it  is 

12 

6 

0 

201.  Writes  poorly 

16 

2 

0 

Decision  Making 

202. Delays  decisions  unnecessarily 

4 

12 

2 

203. Generates  innovative  solutions  to  unique  problems 

12 

6 

0 

204. Ignores  information  that  conflicts  with  own  assumptions 

5 

12 

1 

205. Makes  sound  decisions  in  a  timely  manner 

9 

9 

0 

206.  Willing  to  revisit  a  decision  when  new  information  calls 
for  it 

7 

11 

0 

Motivating 

207. Creates  a  supportive  work  environment 

13 

4 

1 

208. Disciplines  in  a  firm,  fair,  and  consistent  manner 

5 

1 

12 

209. Inspires  people  to  do  their  best 

13 

5 

0 

210. Often  acknowledges  good  performance  of  others 

15 

3 

0 

21 1.  Sets  clear  performance  expectations 

7 

4 

7 

Developing 

212. Does  not  encourage  professional  growth 

1 

10 

7 

2 13. Is  an  effective  teacher 

5 

8 

5 

214. Often  uses  counseling  to  provide  performance  feedback 

3 

3 

12 

215. Provides  opportunities  to  learn 

4 

7 

7 

216. Seldom  delegates  authority 

6 

4 

8 

Building 

217.  Actively  participates  in  organizational/unit  activities 

18 

0 

0 

218. Encourages  cooperation  among  team  members 

16 

2 

0 

219. Encourages  organizational/unit  activities 

17 

1 

0 

220.Eocuses  the  organization/unit  on  mission  accomplishment 

11 

7 

0 

221. Treats  others  as  valuable  team  members 

17 

1 

0 

Learning 

222. Becomes  defensive  when  given  critical  feedback 

15 

3 

0 

223. Encourages  open  discussion  to  improve  the 
organization/unit 

13 

4 

1 

224. Helps  organization/unit  adapt  to  changing  circumstances 

10 

8 

0 

225. Seems  to  be  realistic  about  own  personal  limitations 

9 

9 

0 

226.  Willingly  accepts  new  challenges 

14 

4 

0 

Planning  and  Organizing 

227.  Anticipates  how  different  plans  will  look  when  executed 

3 

9 

6 

228. Develops  effective  plans  to  achieve  organizational  goals 

7 

9 

2 
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229. Leaves  key  events  to  chance 

4 

9 

5 

230.  Sets  clear  priorities 

14 

4 

0 

231.Lfnwilling  to  modify  original  plan  when  circumstances 
change 

11 

4 

3 

Observable 

Somewhat 

Observable 

Not 

Applicable 

Executing 

232. Completes  assigned  missions  to  standard 

14 

4 

0 

233. Does  not  meet  mission  timeliness 

14 

4 

0 

234. Does  whatever  is  necessary  (within  ethical  limits)  to 
complete  the  mission 

10 

8 

0 

235. Monitors  execution  of  plans  to  identify  problems 

6 

9 

3 

236. Refines  plans  to  exploit  unforeseen  opportunities 

6 

8 

4 

Assessing 

237. Accurately  assesses  the  organization/unit’s  strengths 

4 

9 

5 

23 8. Accurately  assesses  the  organization/unit’s  weaknesses 

4 

9 

5 

239. Makes  organizational  changes  for  no  apparent  reason 

1 

5 

12 

240. Rarely  conducts  after-action  reviews 

3 

2 

13 

241. Takes  time  to  find  out  what  subordinates  are  doing 

5 

5 

8 

Respect 

242.  Actively  supports  equal  opportunity  for  all  persons 

13 

4 

1 

243. Creates  a  climate  of  fairness  in  the  organization/unit 

11 

7 

0 

244. Excludes  some  from  team  activities 

13 

5 

0 

245. Treats  others  with  respect 

18 

0 

0 

Selfless  Service 

246. Claims  credit  for  other’s  work 

14 

4 

0 

247. Considers  the  needs  of  own  and  others’  family  members 

9 

9 

0 

248. Places  the  welfare  of  the  organization  before  own 
personal  gain 

11 

5 

2 

249. Takes  advantage  of  others  to  advance  own  career 

9 

9 

0 

250. Takes  privileges  not  allowed  others 

8 

6 

4 

Integrity 

251. Behaves  with  questionable  ethics 

14 

3 

1 

252. Demonstrates  moral  courage  (does  what  is  right) 

12 

6 

0 

253.1s  not  sensitive  to  the  ethical  impacts  of  decisions 

10 

6 

2 

254.1s  trustworthy 

13 

5 

0 

255. Sets  the  proper  ethical  example  for  others 

12 

6 

0 

Emotional  Stability 

256. Does  not  display  extreme  anger 

17 

1 

0 

257. Exhibits  wide  mood  swings 

17 

1 

0 

258. Maintains  calm  disposition  under  stress 

16 

2 

0 

259. Possesses  an  even  temperament 

18 

0 

0 

260.  Seems  to  behave  unpredictably 

17 

1 

0 

Other 

72.  Demonstrates  appropriate  warrior  skills 

9 

6 

3 

73.  Is  a  clear  thinker 

15 

3 

0 

83.  Is  effective  of  the  job 

14 

3 

1 

84.  Maintains  effective  interpersonal  relations  with  others 

17 

1 

0 

85.  Physically  fit  for  the  job 

14 

4 

0 

86.  This  person  is  a  good  leader 

14 

3 

1 

87.  This  person  is  someone  I  would  follow  into  combat. 

12 

5 

1 
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Appendix  L 


Research  Question;  Who  should  measure 
“Who  do  you  believe  could  contribute  feedback  on  those  traits,  skills,  or 
behaviors  listed  earlier?  What  weighting  should  that  feedback  be 

given?” 


Seminar  Leaders 

Weighting 

Peers  Only 

a 

Peers- 100% 

e 

Peers- 100% 

Peers  and  CIs 

g 

Peers-50%,  CIs-50% 

Peers  and  Ops 

d 

Peers-60%,  Ops-40% 

Peers,  CIs,  Ops  Officers 

Peers,  CIs,  Ops  Officers,  and  Senior 

b 

Peers-65%,  CIs-25%,  (Ops  and  Senior  Leadership)- 15% 

Leaders 

h 

Peers-35%,  CIs-35%,  Ops-20%,  Senior  Leadership- 10% 

f 

Keep  current  system  (Peers-16.2%,  CIs-48.5%,  Ops-17.6%,  Senior  Leaders- 17. 6%) 

i 

Peers-25%,  CIs-25%,  Ops-25%,  Senior  Leaders-25% 

CIs  and  Ops  Officers 

c 

CIs-50%,  Ops-50% 

Other 

Commandant/Operations  Officers 

Peers  Only 

Peers  and  CIs 

Peers,  CIs,  Ops  Officers 

c 

Peers-40%,  CIs-40%,  Ops-20% 

Peers,  CIs,  Ops  Officers,  and  Senior 

B 

Keep  current  system  (Peers-16.2%,  CIs-48.5%,  Ops-17.6%,  Senior  Leaders- 17.6%) 

Leaders 

D 

Keep  current  system  (Peers-16.2%,  CIs-48.5%,  Ops-17.6%,  Senior  Leaders- 17.6%) 

E 

Peers-30%,  CIs-30%,  Ops-20%,  Senior  Leaders-20% 

CIs  and  Ops  Officers 

Other  (Peers,  Seminar  Leaders,  CIs, 

A 

Peers-15%,  CIs-20%,  Seminar  Leaders-30%,  Senior  Leaders-35% 

Senior  Leaders) 

Academics 

Peers  Only 

Peers  and  CIs 

4 

Peers-50%,  CIs-50% 

Peers,  CIs,  Ops  Officers 

Peers,  CIs,  Ops  Officers,  and  Senior 

1 

Peers-35%,  CIs-35%,  Ops-20%,  Senior  leaders-10% 

Leaders 

CIs  and  Ops  Officers 

Other  (Ops  and  Permanent  Faculty 

2 

Permanent  Faculty  Member-50%,  Ops-50% 

Representative) 

Other  (Peers,  Seminar  Leaders,  and  CIs) 

3 

Peers-33%,  Seminar  Leaders-33%,  CIs-33% 
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Appendix  M 


Points  of  Contact 


Center  for  Army  Leadership,  Ft  Leavenworth  KS 
Web  Site:  WWW-CGSC.Army.Mil/CAL 

FM  22-100 
LTC  Jon  Smidt 

E-mail  Address:  FM22 100 @ LEA V-EMHl. Army. Mil 
Phone:  Comm  (913)  758-3562,  DSN  585-3562 

Leader  Azimuth  Check  Inventory 
LTC  Craig  Bullis 

E-mail  Address:  BullisR@LEAV-EMHl.Army.Mil 
Phone:  Comm  (913)  758-3245,  DSN  585-3245 

Army  Research  Institute,  Ft  Leavenworth  Field  Office 
Dr  Stanly  Halpin 

E-mail  Address :  HalpinS  @  LEA V -EMH 1  .Army .Mil 
Phone:  Comm  (913)  684-9758 
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